So, that being said, lets get to my first post.
So, with the new fall TV season's starting up this week, the past week has seen me paying some serious attention to my PVR setup. First a little background, then I'll get to the latest developments and where I plan on going from here moving forward.
The background:
Several years ago one of my wife's massage clients turned me toward a Linux based Video recording system that worked simular to a Tivo, but without the cost and with many more features. The software he was using and pointed me toward was known as KnoppMyth. The system intrigued me, and before long I was searching for a tv tuner that I could put into a system to get this up and running for myself. I eventually stumbled on a system that the owner no longer wanted (would cost more to fix than buying a new system) that had a tuner in it. I was able to use that tuner and after much tweaking in the system figured out the settings needed to make it work. Truth be told, if I wasn't such a geek I don't think I'd have stuck with it as long as I did with how long it took to get this "cheap" software tuner to work in Linux. However I did, and we started recording. I'm not sure what version of KnoppMyth this was, but eventually I upgraded to a newer version (R5F27) to get some better features, and had to re-tweak the upgrade to make it work yet again. We grew to love the features the system had introduced us to: commercial skipping, gaming (Emulation specifically), web interface for searching/scheduling recordings, not dealing with VHS tapes, just to name a few.
So, this system served us well for a time, with a 120 Gig drive, I eventually realized if we wanted to keep any of our recordings long term I'd need to seriously consider a larger drive. So, found a deal on a 250 Gig Drive and used some tech tools (I'll cover these in more detail in another post) to copy our data and settings from the smaller to larger drive, and got everything up and running with more storage space. About this time our first child was also getting around more and as curiosity would have it, we had to keep an eye out that she wouldn't turn the computer off as it was in our living room. This lead me to look further into setting up a frontend/backend setup where the system doing the recordings etc would be somewhere out of the way and somewhere it wouldn't get shut down, and then another system would serve as the viewing portion in our living room, where if it got shut down and we didn't notice it it wouldn't keep recordings from happening. As time moved on we started to find we were having some conflicts with what to record and when. So, I started looking to upgrade the system, this time with a "hardware" based tuner since prices had dropped since I first set the system up. The search turned to Ebay where I found a Hauppauge PVR-150 for around $20 (If I recall correctly). I snapped it up as it was supposed to be supported out of the box by KnoppMyth (ie no having to tweak with drivers/firmware etc). When I received it I attempted to use it in my production box first and it didn't come right up which confused me, so I pulled it to play with in a test system while my production box kept recording away as it had been. After some testing in a KnoppMyth test box as well as my Windows PC, in which it didn't work in either case, I took a closer look at the card and noticed that some of the surface mount components were missing/scratched off. Ebay auction was "as-is" so no recourse there. Back on the search, I found a PVR-500 card for around $40 shipped (on sale), however it did not come with the PCI mounting bracket. After some research I decided that the card connections looked close enough to those on the bad PVR-150 that I figured I could make it work even if I had to modify it slightly. Added benefit of this card is it has two hardware tuners built in, so it by itself could record two simultaneous programs by itself, and I still have the "software" based card as well. When the card arrived I first put it into test system and found it worked as expected. Big time Woohoo.
So, now we were able to record up to 3 shows at the same time, and the drive did get to the point where I realized it would be too small again. Found a good deal on a 500 Gig drive and made the purchase. That was several months ago, and I just never got around to doing the copy and making the switch, not until recently anyway.
Wow, I just realized that whole history could have made an entire post by itself, but think it's relevant enough to keep here so you know what all has been put into the system to date (though I do believe it skimmed over some of the more difficult parts, they took place so long ago I don't feel it would do justice to go into more detail from memory at this point.).
The Latest Developments:
So, about 2 months ago (oh, no, more history) we had a branch fall which took out the cable line to our house. When Comcast came out to repair it (broke at the pole) the technician put on a filter which would allow us to get extended basic for a period of time until they finished doing their upgrade to "digital" for the extended basic programming. This in effect gave us twice the content to record and view. The 250 Gig drive filled QUICKLY, and in fact we ended up deleting some recordings without ever watching them. Then Comcast did their magic, and we are now back to our Basic BASIC cable, but still sitting there with not a lot of free drive space, and the upcoming fall season shows about to start.
So, about two weeks ago I decided to move up to the 500 Gig drive. As mentioned earlier I had had success before using some software tools, and I decided to go that route again, though I didn't remember exactly which tools I had used, I went with my tried and true route. Symantec Ghost was the tool I booted to first to "clone" the drive, settings, data and all. Well, the first attempt failed with errors. OK, lets try it with an option in the software to continue even when errors are encountered. Second attempt failed. Ok, something isn't right here, but can't put my finger on it exactly. So, I move on to another tool I had come across, probably about the time I did this before, (not sure which I had used before successfully) Clonezilla. So, I burn the bootable disk for Clonezilla and go through the process. Things were looking good and I had other tasks to do around the house so I let it run. After some time I came back and it failed as well. So, reboot and go back in and tell it to ignore errors/force the clone and walk away again. By the way this was over a Friday evening and part of Saturday during the day. Well, that also failed. At this point I took a closer look at the errors and used Google to research, looks like the drive has some physical errors on it that are causing the clones to fail. This isn't good, but the system will run as it is with the 250 Gig drive even with the errors, and I was running out of time in my window of no recordings. So, I put the 250 Gig drive back in and bring the system up so that we can continue on as before. This also gives some time to think through how move forward with moving to the larger drive.
Couple issues that stuck in mind as I contemplated where to go from where I was at. One, the drive has physical errors that are stopping duplication, how long till those errors cause problems with the system overall? How do I get things over to the drive seeing as all clone utilities fail with the physical errors, yet the system is able to run and appears to work fine? How long would the system hold up if I just used the 500 Gig drive as a data mount, keeping the 250 Gig as just the OS and utilities portion, which isn't good if the 250 Gig drive completely fails? These questions really had me wondering. So, in my research I looked further into how the tools Clonezilla uses to perform it's functions (really the benefit of Clonezilla is that it is a pretty wrapping on tools that are commonly available, and makes it easy to step through the process of using those tools). I decided that maybe it would be best to just create the file system on the new drive then copy the data over manually using other tools at my disposal in the operating system.
So, I burn a disk of GParted, another open source package that gives a nice GUI for functions that can be done from the command line, which is very simular in functionality to Partition Magic. Using GParted I created the partitions on the new drive and then found it (GParted) contained a "copy" feature. I used this copy feature and it copied over all of our recordings without a problem. This was good news, in that at least the recordings would be there, even if they were not accessible through the "recordings" menu in KnoppMyth. After it finished copying the recordings I decided to give a try copying the system partition, and after a time it came back with an error while copying. Not good. So after some research I found that I could try to use rsync (a Linux copy utility) to try manually copying the data. Gave that a try and it seemed to be taking a very long time so I stopped it. At this point I realized I messed up the command sequence and I was copying data to the location I was copying from (basically creating a loop which would have eventually filled the drive). So after cleaning up my mess, I started the process again, and rsync came back with errors, but said it was able to copy most everything. Woohoo!
At this point I need to get the system back up as soon as I can, so I pull the drives and switch which is Master/Slave so that I can make sure that the system will boot off of the new drive. Power it up and am greeted by "L 99 99 99 99 99 99..." filling the screen. Oh, No.. Off to my friend Google and look up ways to resolve. Lots of information out there, and lots that didn't pertain to me. So, I go to the KnoppMyth forums/wiki pages and start research. Found a couple of different fixes to try, one of which utilizes a rescue boot floppy, guess what, I didn't have one. After some more digging I found a post where a user was able to boot from the install cd for KnoppMyth, and mount the file system, edit some files and rebuild the boot record. Well, this is where I find my first problem, the file on the new drive isn't working, so I then mount the second drive and copy the file from there. After some tweaking, I was able to rebuild the boot record. Somewhere in there I also did another rsync copy of the old drive to the new drive in hopes it would fix it as well, but it didn't, but need to mention I did do that again and it came back and said there were files it was unable to copy (physical errors again). So, I reboot and the system comes up, but not fully, KnoppMyth frontend (the user interface for using the system) didn't come up. After some quick research I found what files were needed to get that back up and copy them from the old drive. For some reason these didn't copy right when doing the mass copy, but did with the smaller copy. So, now the system is up and appears to be running correctly, so I went upstairs and made sure that the Frontend system was also working correctly. It appeared to be and it was late so off to bed I went.
Fast forward a week to this past weekend when I got to thinking it would be nice to get closed captions working on the frontend system (they worked on the backend in the basement, but not in the living room), and also as I was looking at recording job options the "Encode for iPod" caught my eye again. This latest caught my attention as I was trying to figure out when I'd get time to watch shows that my wife wasn't interested in, and I can't watch them while our daughter is watching her shows.
Well, the Closed captioning search lead me to CCExtractor, a free tool that reads through the original recording and creates a text (srt) file of the captions. So, I download the tool, and install it, but it errors out on me, and I eventually figure out the syntax needed to make it run and generate the file. After I've generated a caption file on the backend I went upstairs and verified that when viewing that recording captions are now working. WooHoo!! After that I created it as one of my user jobs with proper syntax, so that when I set to record a program, it is just a matter of checking one extra box and the Closed Captions will be extracted and usable on the frontend. After all that I was thinking last night about how things were set up between the frontend and backend, and how there might be differences. You see, with this system there are a couple of options for programs to use when viewing videos/recordings. The main two video players are ffmpeg and mplayer. So I was thinking perhaps the reason the captions were working on the backend but not the frontend were that they were using different video players and the one being used on the frontend just wasn't able to read the encoded captions (but was able to load the srt file of extracted captions). So, this morning I went through the settings and did find that the backend used ffmpeg and the frontend was set to use mplayer, so I changed the settings so the frontend is now using the ffmpeg. This change was not able to do away with the CCExtractor tool yet though.
Now in the middle of all of that I also, as mentioned, was looking at the "Encode to iPod" option. This really interested me in that we both have iPod touches, and in the future I plan on moving on to an iPhone. So, I started by looking into how this worked on the backend, and found the scripts used to create the file for the iPod. I wasn't 100% sure this would work as my version of KnoppMyth is rather old, actually from before the iPod touches had really hit the market, and at the time it was just the iPod video that the scripts were set up for. So after some searching on the KnoppMyth forums I found a user who had changed the scripts to make a file that looked good and worked well on his iPhone (I'm assuming a first or second generation iPhone as he stated he was running the 1.1.1 OS on the iPhone). Well I duplicated his settings in the myth2ipod script, but when I ran the script on a file I received errors for all my efforts. After more searching I found a "patch" that a user had applied to one of their setting files, but when I tried to run the patch according to instructions it failed, and no massaging the files helped any. After some time I started to look at the files to see what the contents were to see if I could manually patch it, and found that the file it was trying to patch was empty. So, I looked at the file sized for all the files in that directory and found they were all 0 bytes in size. Drat, now what. I started to think I'd be up the creek, having to whittle my paddle out of another system set up just to retrieve the files from, when I realized: The frontend system should have these files un-molested. So, off I went (over the network) and sure enough there they were full of script goodness. So, I copied the files over the network, and tried the script again, FAIL. Found even more files that were "missing" but not missing, so replaced them, this time it was only about half the files in the folder, but I replaced all of them anyway. Try again.. wait, what is this, it's working.. WooHoo! Oh, hold that excitement a little, lets wait till it is completely done, and actually on my iPod before I completely rejoice. So, an hour and a half later it's done encoding it, and now there is a file in the ipodfeed folder, looks good, just a little over 640 meg (hmm.. these could fit on a cd..) Into Itunes I go, subscribe to "Podcast", wait to download. Plug in iPod, transfer to iPod, play. Looks pretty good, nice. Now I can set up my shows to be encoded for the ipod, transfer them and watch my shows over my lunch break or even while our daughter is using the TV. WooHoo..
Moving Forward:
Ok, this post has been entirely longer than I had thought it would be, so this will hopefully be short, and I'm just going to bullet list it for you.
- New Frontend system (mini desktop that will be just slightly larger than say a VCR)
- Set frontend system to boot over network, so no need for power consuming hard drive
- Reformat the 250 Gig drive to hold Videos not recorded off tv (ie VHS converted files)
- Set up system that can be used separately to record VHS tapes for digital access
- Maybe upgrade to a newer version of KnoppMyth/LinHES
In Closing:
Let me first say I doubt very much all of my posts will be this lengthy, in fact I hope they are not. This post covered a long period of time invested in this system, but had background that I felt would help in understanding the overall picture. This post also just seemed to flow out as the excitement I have over getting these things working needed expressed in some manner.
If you would like more information on any part of this post to be covered in more detail in the future, be sure to comment and I'll see about creating specific posts which will cover those details. (IE if you are interested in a review of the system with a better list of features and maybe screen shots (if I can figure out a good quality way to do so) let me know.) I may cover some of the tools mentioned in more detail at some point anyway, but will do so sooner if anyone expresses any interest. I will probably also keep updates coming as I make changes to the system expressed in the "Moving Forward" section.
