Revisiting my Backup Strategy

I just got back from picking up the results of an expensive data recovery job from a client’s failed hard drive. It reminded me that I hadn’t updated my own backup strategy in a while. I recently changed it after learning a few valuable lessons and being very lucky after a device failure.

Below is a comment I left on a photography podcast that I listened to recently. It describes the work flow I am currently using. While it deals with my image back ups, I do cover off other files.

After a few months of using this workflow, I can’t say enough about the Bestsync Software mentioned. It is available at www.risefly.com. I have also added another tool recently. I am using Sugar Sync to do some offline folder backups as well. This seems to work very well and I can choose what ever folders that I want to backup. You can download and get 5 Free Gigabytes of online storage. In addition using this link https://www.sugarsync.com/referral?rf=907xn2ham8fg&utm_source=website&utm_medium=web&utm_campaign=referral you will get an additional 500 MB of free space as well as giving me an additional 500 MB!

I am still using Dropbox as well and they have also just boosted the free amount of online storage you can get. Again sign up and install using this link and we both get bonus space free. http://db.tt/Rz3VAiH.

Use both links above and you will get 8GB of free online storage!

Posted on photography.ca

Marko,
I feel your pain. As a small business IT Consultant I deal with the Data Recovery companies and failed hard drives every couple of months. As a photographer I have also been burned either by not being ready or the “I’ll do it later” syndrome.
A couple of things to note. RAID is not the only answer. Yes they are redundant but they can fail too. I have experienced failures with Drobo’s and other RAID devices as well.
I have developed my own back up system that works pretty well keeping in mind that if the system is not automated, it will not get done.
In our home I have all of the PC’s connected to a first Gen Windows Home Sever so my kids and wife’s system all get automatically backed up nightly to that box (not an answer for catastrophe though).
For myself I use the following.
When I bring images in I use Lightroom to import and copy all of my images to a folder on my E-Sata connected Drobo S (5 x 1TB hard drives). My Lightroom catalog files also reside in one folder on the Drobo including the Lightroom backup files. All files are Canon CR2 Raw.
Nightly I have a program running on my system called Bestsync 2012 that synchronizes the Lightroom catalogs and the image files from the Drobo folders to a Mediasonic Raid 1 enclosure connected via USB 3 with 2 x 2TB Drives. I also have Best Sync set up to synchronize the Mediasonic backups to one of 2 USB External USB drives when the are plugged in to the system as well as on a nightly schedule. Every couple of weeks I swap these drives and take them to my parents house where I connect to them to a Pogoplug so I can access them across the net in case there is any need.
The stuff I show off and want to share with family and friends I put up in galleries at Smugmug (power user account). I have unlimited Storage there and they take large high quality jpg files plus I like their galleries.
I chose Best Sync because it wasn’t that expensive, Pro License was 38.00 USD and it offered VSS (Volume Shadow Service) so that open files are synced in case I am still working on them or left Lightroom open accidentally. You can set up tasks to do almost anything you need and have multiple different tasks and different destinations going at different schedules. For example I sync Music files to my home server as well as Outlook PST files and Quicken files. In addition BestSync offers compression, encryption and a backup vault for files that get deleted on the source folder. These are backed up from the target folder before being deleted there in case the deletion was accidental.
I also use Windows Live Mesh to sync some business documents to the cloud and some smaller stuff I use Dropbox and box.com.
Best Sync was the way I decided to go in October after coming down one morning to find my Drobo reporting as a RAW drive (talk about heart failure). I used Zero Assumption Recovery to get all the data off that drive (it took about 60 hours) reformatted the Drobo and it has been solid since and even correctly reported (and protected data) from 2 failed hard drives in a 2 week span (all from the same batch at Seagate). Although I am not completely sold on the Drobo any more and may look for another Raid 5 solution.
Another free program you can use is Microsoft’s Sync toy. It can be scheduled to run as a scheduled task as well but doesn’t support VSS. But it can mirror or sync from one drive to another.
One additional thing I was going to start doing at the cost of some additional storage space was to save Lightroom adjustments as XMP sidecar files. Doing this would allow them to be reimported into a fresh copy of Lightroom with all of the adjustments made available.
For travel I store images on a laptop hard drive, then to 2 external hard drives and then to a hyperdrive UDMA color drive. I also try to avoid deleting images off of CF cards until I return home and transfer all of my images from my laptop into my backup system via a Lightroom Catalog export from the laptop.
There are some great new appliances coming out in the next little while as well built on Windows Storage Server Essentials that will allow for NAS and client backup. I saw one from Western Digital back in October that looked very promising.
While my backup system may sound confusing it is all accomplished by the Best Sync Software. I have found it very flexible and I can tailor it for all of my own uses.
Stephen Kennedy
Calgary, Ab, Canada

Upgrading your Shaw HD PVR

I was searching the web recently trying to figure out how to move recorded content from my Shaw HD PVR to my DVD recorder via firewire. While searching I saw that many Shaw users were indicating that their Motorola HD PVRs had received the firmware that enabled the eSata port on the back of their boxes. More searching indicated that the while the firmware could activate the port it was not yet active. Forum posts indicated that Shaw had told several people that the ports would be activated soon.

Last week while checking out the Shaw site, I saw that the Shaw was now selling a PVR expander for 200.00 CAD. This PVR expander is a Western Digital external 1TB eSata drive. I called Shaw to find out if you needed the Shaw PVR expander or could you use any external eSata drive.

Update November 13, 2009 – I just found out that Shaw has dropped the price of their PVR Expander to 150.00 CAD. TIt looks like they now have the Western Digital drive and a Seagate Drive Available.
I was told that you could use any drive, and that the eSata ports on the Motorola boxes were globally enabled at the end of October.

I ran down to Memory Express (http://www.memoryexpress.com) where I picked up a Velocity external USB/eSata enclosure and a Seagate 1 Terabyte drive for 119.00.
Once I got home it took me about 15 minutes to install the drive in the enclosure and then another 5 minutes to plug it into my Shaw box’s eSata port and get it powered up. When I fired up the Shaw box it indicated that I was not authorized to have an external drive. I 10 minute call to Shaw Technical support and the enabled the port on my box and rebooted it.

When the box came back on it indicated that the drive was not formatted for the PVR and did I want to format it? I indicated yes and within a couple of minutes I flipped to the PVR menu. Sure enough all of our old recordings were there however there was a nice number 0% of space used. we have since recorded several shows and space is no longer an issue.

So I upgraded the box in less than half an hour with a new external drive and now we don’t have to worry about space. I haven’t tried this yet but I should be able to plug in yet a different eSata drive if I ever need more space. I should be able to swap between a couple of drives over time.

Shaw indicated that they wouldn’t support a drive other than the one they are selling however it is a very easy procedure. One problem with the Velocity enclosure is that it is a little noisy compared to the Western Digital firewire enclosure I have but it is not too bad. If you are looking at a little quieter solution Costco in Calgary is selling a 1 TB external drive with USB, Firewire and eSata for 159.99. So it is still 40.00 cheaper than the Shaw drive. See below for updates. There is not really much to performing this upgrade. If your port is not activated it is a simple call to Shaw support asking them to activate it which requires a box reboot and you lose your programming guide and menus for a short period of time, but it truly is a plug and play install.

I have read that the largest drive supported is 1 TB but I haven’t confirmed that as of yet but better to stay with the 1 TB size. It will give you significantly more recording space than the default 160 GB drive that comes with the box.

Important Update November 19th, 2009

I have been reading the forums and have discovered a few things about upgrading your Shaw PVR with an external hard drive enclosure.
1) There is a problem with external enclosures that feature a sleep or low power setting (most of the desktop style drives from Western Digital and Seagate). These drives will go to sleep and not work with the PVR. The lower costs enclosures don’t do this nor do the dedicated PVR Expanders. I would suggest staying away from the Costco WD drive mentioned above.

2) There is a known problem where your Motorola Box will not be able to track space on the external drive and will always register at 0% full. Some people have had Shaw fix this by resetting their boxes several times however another reset (even a firmware upgrade or a power failure will reset this down to 0 again). When the box starts tracking again it will ignore what content is on the drive and start counting at 0 again not indicating how much space there is. To check you use the diagnostics. Power off your box and power it on hitting select immediately, go to PVR diagnostics and use option 13. The first two numbers on each drive will tell you space available on the drive in GB. Motorola will have to release new firmware to connect this and it looks like it might be spring of 2010 before seeing it. This happens with all drives attached.

3) If your PVR box is connected to your TV via an HDMI cable, some people have reported that they do not see the format option. Connect your PVR via component or composite cabling to the TV  to set up the drive

4) Once a drive is connected to the PVR it cannot be swapped. Contents are encrypted and tied to the box they are connected to. Swapping drives will not work as you will not be able to get your old content back after changing external drives.

5) The enclosures that Shaw sells contain special drives designed to record video. These are non error correcting drives (unlike desktop drives). I have had no issues with my EC drive but some people have reported video and audio dropouts in recorded shows and this can be caused by built in error correcting on the drives. With the lower price being offered by Shaw on their expanders (in addition to a 3 year swap warranty) I would suggest that these drives are the best way to go. I may soon be switching my drive to one of the Shaw Expanders.

Didn’t Follow My Own Rules!

Well yesterday I didn’t follow my own backup rules and got burned! On my main desktop I had a 500 GB Seagate drive connected via an eSata dock to the system. I was trying to copy files to this drive (an XP CD and SP3) to make a bootable XPSP3 CD for a client when I started to get drive errors!

Eventually I couldn’t access the drive at all. I tried several things to get access to the drive including several data recovery software packages and had no luck. While most of the data on this drive was backed up folders or data that I had backed up in another location, there was one folder containing pictures that I had shot of both my son’s and daughters soccer teams in various games and tournaments this season. In total there was about 3500 photos there. Unfortunately I didn’t back up this folder anywhere (against my own rules)!

Regular readers will know that I am an Adobe Lightroom user and luckily I didn’t store the catalogue for these photos in the same folder (they are on my main drive and yes these are backed up regularly to my various NAS devices).

I opened the Lightroom and could see that I had the previews for the images in the database, and so I searched the web to see if these could be extracted to jpg files. I ran across this post http://lightroomsecrets.com/2009/05/recover-lost-images-with-lrviewer/ which in turn led me to this software http://basepath.com/ImageIngester/LRViewer-info.php. I downloaded the software, installed it and was able to save all of the preview images as jpg files. Earlier today I found that my favorite Lightroom plugin author also has a similar plug in on his site that is available here http://regex.info/blog/lightroom-goodies/preview-extraction.

While these are lower resolution files than what I originally had at least for posting to the web and allowing the parents to print 4 x 6 images they will be fine. Another thing I noticed is that not all of the preview files were full sized. Most of the images that I had marked for export were in a larger size however there are a few images that were a little smaller.

Thanks to these two tools, I was able to get back some of the images that I wanted to share with others.

All in all I learned my lesson yet again. I will remember this problem and have already set up MS Sync Toy to run as a scheduled task on my system and backup all photo related folders to my Drobo on the DroboShare or my HP WHS.

Once again I hope that this is a lesson learned for anyone about how easily it is to lose data if you don’t back up properly!