Revisiting my Backup Strategy

I just got back from picking up the results of an expensive data recovery job from a client’s failed hard drive. It reminded me that I hadn’t updated my own backup strategy in a while. I recently changed it after learning a few valuable lessons and being very lucky after a device failure.

Below is a comment I left on a photography podcast that I listened to recently. It describes the work flow I am currently using. While it deals with my image back ups, I do cover off other files.

After a few months of using this workflow, I can’t say enough about the Bestsync Software mentioned. It is available at www.risefly.com. I have also added another tool recently. I am using Sugar Sync to do some offline folder backups as well. This seems to work very well and I can choose what ever folders that I want to backup. You can download and get 5 Free Gigabytes of online storage. In addition using this link https://www.sugarsync.com/referral?rf=907xn2ham8fg&utm_source=website&utm_medium=web&utm_campaign=referral you will get an additional 500 MB of free space as well as giving me an additional 500 MB!

I am still using Dropbox as well and they have also just boosted the free amount of online storage you can get. Again sign up and install using this link and we both get bonus space free. http://db.tt/Rz3VAiH.

Use both links above and you will get 8GB of free online storage!

Posted on photography.ca

Marko,
I feel your pain. As a small business IT Consultant I deal with the Data Recovery companies and failed hard drives every couple of months. As a photographer I have also been burned either by not being ready or the “I’ll do it later” syndrome.
A couple of things to note. RAID is not the only answer. Yes they are redundant but they can fail too. I have experienced failures with Drobo’s and other RAID devices as well.
I have developed my own back up system that works pretty well keeping in mind that if the system is not automated, it will not get done.
In our home I have all of the PC’s connected to a first Gen Windows Home Sever so my kids and wife’s system all get automatically backed up nightly to that box (not an answer for catastrophe though).
For myself I use the following.
When I bring images in I use Lightroom to import and copy all of my images to a folder on my E-Sata connected Drobo S (5 x 1TB hard drives). My Lightroom catalog files also reside in one folder on the Drobo including the Lightroom backup files. All files are Canon CR2 Raw.
Nightly I have a program running on my system called Bestsync 2012 that synchronizes the Lightroom catalogs and the image files from the Drobo folders to a Mediasonic Raid 1 enclosure connected via USB 3 with 2 x 2TB Drives. I also have Best Sync set up to synchronize the Mediasonic backups to one of 2 USB External USB drives when the are plugged in to the system as well as on a nightly schedule. Every couple of weeks I swap these drives and take them to my parents house where I connect to them to a Pogoplug so I can access them across the net in case there is any need.
The stuff I show off and want to share with family and friends I put up in galleries at Smugmug (power user account). I have unlimited Storage there and they take large high quality jpg files plus I like their galleries.
I chose Best Sync because it wasn’t that expensive, Pro License was 38.00 USD and it offered VSS (Volume Shadow Service) so that open files are synced in case I am still working on them or left Lightroom open accidentally. You can set up tasks to do almost anything you need and have multiple different tasks and different destinations going at different schedules. For example I sync Music files to my home server as well as Outlook PST files and Quicken files. In addition BestSync offers compression, encryption and a backup vault for files that get deleted on the source folder. These are backed up from the target folder before being deleted there in case the deletion was accidental.
I also use Windows Live Mesh to sync some business documents to the cloud and some smaller stuff I use Dropbox and box.com.
Best Sync was the way I decided to go in October after coming down one morning to find my Drobo reporting as a RAW drive (talk about heart failure). I used Zero Assumption Recovery to get all the data off that drive (it took about 60 hours) reformatted the Drobo and it has been solid since and even correctly reported (and protected data) from 2 failed hard drives in a 2 week span (all from the same batch at Seagate). Although I am not completely sold on the Drobo any more and may look for another Raid 5 solution.
Another free program you can use is Microsoft’s Sync toy. It can be scheduled to run as a scheduled task as well but doesn’t support VSS. But it can mirror or sync from one drive to another.
One additional thing I was going to start doing at the cost of some additional storage space was to save Lightroom adjustments as XMP sidecar files. Doing this would allow them to be reimported into a fresh copy of Lightroom with all of the adjustments made available.
For travel I store images on a laptop hard drive, then to 2 external hard drives and then to a hyperdrive UDMA color drive. I also try to avoid deleting images off of CF cards until I return home and transfer all of my images from my laptop into my backup system via a Lightroom Catalog export from the laptop.
There are some great new appliances coming out in the next little while as well built on Windows Storage Server Essentials that will allow for NAS and client backup. I saw one from Western Digital back in October that looked very promising.
While my backup system may sound confusing it is all accomplished by the Best Sync Software. I have found it very flexible and I can tailor it for all of my own uses.
Stephen Kennedy
Calgary, Ab, Canada

Back Up Primer for Home Users – Part 1 – Hardware

I have posted similar articles in the past on this subject, but it seems that every 3 to 4 months, I end up working on a system that has crashed and the user did not do a backup!

Your PC or Mac can crash due to several different reasons; Malware and Viruses, Hardware failure (most typically Hard Drives) and Operating System crashes. A lot of clients say that they aren’t really concerned about their data until they learn that they can’t just re-download music from iTunes and that all of the digital photos they have taken for the last three years are gone. That’s when they panic!

Not every crash results in a loss of data, for example, a corrupted registry problem can be easily fixed and the data is easily recoverable from the system. However a physical hard drive crash (the only real mechanical part in a system) is a little different. Data can be recovered but it is costly.

I recently sent a 60GB drive into a Data Recovery Center on behalf of a client where the drive was completely dead. It cost the client 100.00 for the company to look at the drive and report what could be recovered. The cost of the Data Recovery for this drive was 1800.00 although almost all of the data was recoverable! Spending a little bit of time and money can save you from large bills like that if you have critical data on your system.

There are a couple of ways you can back up your system, A full system backup (usually an image based backup) that backs up everything or a data backup where you select only your data to back up. This is the method that I usually use because I can always rebuild my system with the original install disks and reinstall my applications.

Next you have to decide where you are going to backup to. Do you just want to keep a local backup or do you want to protect yourself from catastrophe (Fire, Theft, Flooding) and use an offsite backup or both?

Personally I use several methods to backup. I use a lot of local backups and for very critical data I also use offsite backup. The simplest method of maintaining an offsite backup is to keep a copy of the back up media in a secure place outside of your home (locked in your office desk, at a relative’s house, etc).

What You May Already Have

So what do you need to get started? At the very least you probably already have the most basic backup device already in your system. Virtually every system sold in the past 4 to 5 years will either have a CD Burner or better yet a DVD Burner in it. You can use this burner to backup data to a blank disk and store it some where. I have been using this method myself for a long time. If you only have a CD Burner you can put approximately 700MB of information on a single disk. If you have a DVD Burner you can put approximately 4.7 GB of Data on a disk. DVD disks can be purchased very inexpensively especially if you buy them in Spindles. I will typically look for sales and pick up 100 DVDs for less than 20.00.

While burning to a DVD or a CD is a good idea, but I wouldn’t rely solely on it. Cheap DVDs or CDs typically do not last very long and some can even go bad after even just a couple of years. If you plan on backing up this way for the long term, at least plan on buying Archival quality disks. These disks are designed to last more than a few years.

One advantage of backing up on DVD is that you can move the disks to another location to give you the added security of offsite storage. I buy inexpensive generic CD binders and keep copies of almost everything in these binders. Multiple copies can also be made and are ideal to keep a set  locked in a drawer at your office or at a relative’s house.

When I used to sell custom PCs, I usually suggested to my clients to build the system with a second drive installed in the machine. With this setup a client could store data or sync data on to the second drive and install programs and the OS to the primary drive. This gave them basic protection against OS Crashes or a hardware failure of the primary drive. Alternatively the client could use the second drive and some imaging software and run full system backups to the second drive on a weekly basis. I still build all of my personal machines with 2 hard drives for this reason. It is not fool proof though as you can lose your data if the 2nd hard drive fails. Some motherboards offer built in Raid controllers as well where Raid 1 can be configured. With this type of a system your primary hard drive is mirrored over to the second hard drive. This is an excellent solution but does make it a touch more difficult to add additional internal storage if needed and has to be set up (ideally) with the purchase of the system.

External Devices

With the popularity of laptops and netbooks today, or if you buy a prebuilt machine and don’t want, or can’t install a second hard disk there are still several options available to you. At the very least, consider a USB key or two. These have dropped in price significantly as well as increasing in capacity over the last little while. You can pick up a couple of 8 GB USB keys for about 40.00 CDN. With these keys you can copy your important documents over to them periodically (or use a little utility I will describe soon) and at least your most important stuff resides in a couple of places. A warning about USB keys however, these devices have a limit to the number of reads and writes they can handle so I don’t suggest relying on them for backup only. Also because they are so small, they can easily be misplaced and or lost and you don’t want your data ending out in the public if lost.

Hard drives have also considerably dropped in price significantly over the years. Along with this external USB drives have dropped in price as well. Some of the brand name drives come with backup software pre-installed on the drives. For example a 500GB Seagate Free Agent Drive sells for about 125.00 CDN and has software that can do backups for you automatically (Windows Only). If you are a Mac user look at a Western Digital My Book drive. Several other manufacturers offer external drives so shop around you may find some at a better price.These are desktop style drives and are not really portable as most require AC power. So if you are a notebook user and want to take the drive with you look at a 2.5” form factor drive. These are available again from a variety of manufactures but are smaller and generally can draw required power from the USB ports on your system. If you are lucky enough to have a FireWire or eSata port on your system consider an external drive that also offer these features. FireWire is as fast as USB 2.0 but is less draining on your system’s CPU during transfers and eSata is as fast as internal hard drives.

You could also make your own external drive by purchasing an enclosure and then buying an OEM hard drive. Currently the sweet spot for hard drives are the 500GB size. You can pick these up in Canada for about 75.00 (OEM Drives). You can pick up an enclosure for 30.00 or so and add the drive. The advantage of doing it this way is if you later want to increase the size of the drive as prices fall on the larger drive sizes you can easily replace the drive you purchased. The same thing can be done for smaller sized notebook (2.5”) hard drives for portable storage.

Personally on my desktop systems and my own and client servers, I have been using USB 2.0/eSata Hard Drive Docks. These are similar to the enclosures above however instead of opening the enclosure up and installing a drive, you can simply plug in a 3.5 or 2.5” Sata drive in the bay and away you go. I typically have these connected via eSata so they are hot swappable and when I want to change drives or one fills up, I eject it and plug in another drive. I have found a manufacturer that also makes Drive Boxes (similar to DVD cases) so when I pop a drive out I stick it in a box and on my shelf. The docks sell for about 50.00 and I found this this is a great way to rotate storage device around.

Another type of external enclosure I highly recommend is the Drobo. While not inexpensive the Drobo can do things that other types of enclosures just can’t do. The Drobo sells for 499.00 for the Firewire 800 / USB 2.0 version and 349.00 for the USB 2.0 only version. This does not come with any hard drives at all, but 4 drive bays. The Drobo is unique because it replicates the same type of functionality as a high end Raid 5 system most often found in servers and enterprises, but in a much simpler and flexible format. Where Raid 5 requires a fair bit of knowledge, hardware and specific hard drive configurations to work, the Drobo makes this very easy.

Raid 5 requires at least 3 hard drives of identical size to implement the original array and increasing the size of the Raid requires backing up all of your data, installing new drives of equal size, reformatting and then restoring data. The Drobo eliminates this completely. With the Drobo you can use up to 4 (minimum of 2) SATA drives of any size in the enclosure and it will give you storage and data redundancy where a part of each drive is used to backup parts of the other drives. With the Drobo if a hard drive fails or get’s full, simply swap it out with a drive of equal or larger size. Currently the Drobo can support up to 4 4GB drives. It also works on both PCs and Macs. You can learn more about the Drobo here.

Due to the amount of photos we take as a family, I currently store most of them on one of two Drobos (1 USB only, 1 Firewire) both filled with 4 500GB drives. Sysguy Consulting is a Drobo Authorized Reseller so if you are interested in seeing a demo of this device or to order please contact us.

Network Attached Storage Devices

If you have multiple PCs on a home network there are a couple of other hardware devices you can consider. For 250.00 you can add a device called the DroboShare to your Drobo unit. This device allows you to connect up to 2 Drobo devices to it and share them on a network via Gigabit Ethernet. You will take a little of a performance hit as the Drobo only connects to the share via USB but it allows you to share this device and take advantage of it’s data redundancies capabilities over your network and by multiple computers. We have recently received our DroboShare unit here in our labs but have not yet had a chance to set it up and test it. The new Drobo Dashboard software program (currently in Beta) comes with a utility called Drobo Copy which will allow you to set up folders and files to be copied or synced to the device.

Another device I can highly recommend is the DLink DNS-323. The DNS-323 is a network connected storage device. It allows you to install up to 2 SATA hard drives in it, and configure them as mirrored drives, a single large hard drive, or as two individual drives. I have been using one of these devices with a 400 GB and a 500 GB drive installed in it for a couple of years now and it has been great. I store things that I need access to from my multiple machines from on it as well as backing up Music and other documents to it. With recent firmware the device also servers as a Media server allowing us to stream Music, Pictures or Video to Xbox 360’s in the house or to the PS3. You can get a DNS-323 for about 160.00 CDN in addition to the cost of adding SATA drives. D-Link is also offering the DNS-323 bigger brother the DNS-343 that holds 4 Sata Drives at an MSRP of 658.00 with no drives.

There are other Network Attached Storage Devices available but many of these do require a little knowledge of Raid systems (unless you buy pre configured) and start at around 750.00 CDN.

Windows Home Servers

If you have multiple systems at home you may also want to consider a Windows Home Server. This is designed to be a headless device (no monitor, keyboard or mouse required), that connects to your home network. Windows Home Servers are available from a few manufacturers or can be custom built for you by a system builder. HP has one of the most popular WHS devices with it’s Media Smart Server line.

The HP Media Smart is a WHS with 4 drive bays, a low end processor and 1 GB of RAM. There are several models available and the most recent also offers support for Mac users (apparently this is coming for owners of the older versions). In the older 1 TB version of the WHS, 2 bays are available for expansion. So you can easily add additional storage to the device. I recently picked up the older 1 TB model of the HP Media Smart Server (EX475) in a clearance sale at half price (350.00). I haven’t set it up yet but soon will be replacing the WHS Beta box I was using as well as upgrading this machine to 2 GB of RAM and 2 additional 500 GB Hard Drives. I will post a complete review after it is set up and running for awhile.

With Windows Home Server there are different ways that you can configure the hard drives. You can have everything run as a single large drive or mirror the drives. Similar to the Drobo you can change the storage size by changing or adding drives. You can also add additional storage via USB and if available eSata docks. You can also set up some dedicated external drives for backup of your WHS device which is new in the latest update from Microsoft.

Once connected and set up (using the web interface) you install the included software on the PCs in your home. This will set up all PCs where the software is loaded to be backed up nightly to the home server box. If the PC crashes, simply insert the Client recovery disk into the PC and you can restore the entire device from the WHS. The only stipulation is that the computers have to be left on during the scheduled back up time in order to be backed up. One of the really great things about the WHS system backup is that it uses a single instance file storage system to save space. What this means is that if you are backing up 2 computers and it sees a file in the first one with a specific date, time and version if when it backs up system two if and sees the identical file on it, it will place a pointer to the file it already has, saving space on the storage device. Your first system backup on the WHS will take a while to complete as it has to back up everything on the system. After the first backup however things should be much faster as it will backup only changed and new files.

The really cool thing about the WHS is that it can do a lot more than just be a Networked Attached Storage Device. The server becomes a web server in your home so that you can access your files and shared files on the server from any Internet Connected PC. If you are using XP Pro or Vista Business or better on your home machine, you can use Remote Desktop and connect and work from your home PC also across the internet (this is a feature borrowed from the MS SBS team).

Because the architecture has an SDK (Software Developers Kit) several third party programmers have written numerous add-ons for the devices that also extend the functionality. There are add-ons that allow you to back up the server offsite (I will discuss this in the next post on backup), use it to stream media in your home, create web pages and photo galleries that friends and families can connect to and more. new add-ons are being created every week for these devices.

You can connect up to 10 client machines up to the WHS so for virtually any home the out of the box configuration works well.

WHS is also perfect for a small office. It gives the users virtually all of the features of a full server (no domain security though) but allows them remote access, backup and shared files.

Sysguy Consulting can custom build a WHS for your home or office as well. Contact us if you are interested.

Conclusion

When I first started this post I had planned on writing everything up. Then after I finished about the hardware required I realized it would be too long for just a single post. I will write a separate post about software you can use for backing up your system next.

While there is a lot of information provided here, I hoped to show that you really don’t need much in the way of hardware to backup your system. You can get the basic hardware for around 100.00 and with some free software I’ll discuss in the next post you can institute an easy back up routing.

If you have any questions please feel free to email us at sysguy at sysguy dot com.