104 — Backing up images like a pro versus a bonehead

Comments

1. Neil Speers says:
   March 8, 2012 at 11:13 pm

   LOL.

   Okay, the sys­tem I set­tled on after years of hard drive and DVD back­ups was to go all hard drive.

   My work­ing files reside on a firewire 2 disk raid drive, which back­ups up daily to a drobo (using back­u­plist+ on the mac), which then gets archived once or twice a week to a series of exter­nal hard dri­ves which travel home with me.

   For archiv­ing I use the West­ern Dig­i­tal Desk­top Dri­ves — which I find more reli­able than the WD portable dri­ves. I used to buy any desk­top drive on sale, but blew up a cou­ple of dri­ves because the power sup­plies looked the same, but appar­ently weren’t. So now I only use one brand so I can leave power sup­plies plugged in at home and at the office and they’re the same.

   I use this sys­tem to backup sev­eral ter­abytes of my por­trait and com­er­cial pho­tog­ra­phy images.

   Reply
2. Stephen Kennedy says:
   March 1, 2012 at 1:37 am

   Marko,
   I feel your pain. As a small busi­ness IT Con­sul­tant I deal with the Data Recov­ery com­pa­nies and failed hard dri­ves every cou­ple of months. As a pho­tog­ra­pher I have also been burned either by not being ready or the “I’ll do it later” syn­drome.
   A cou­ple of things to note. RAID is not the only answer. Yes they are redun­dant but they can fail too. I have expe­ri­enced fail­ures with Drobo’s and other RAID devices as well.
   I have devel­oped my own back up sys­tem that works pretty well keep­ing in mind that if the sys­tem is not auto­mated, it will not get done.
   In our home I have all of the PC’s con­nected to a first Gen Win­dows Home Sever so my kids and wife’s sys­tem all get auto­mat­i­cally backed up nightly to that box (not an answer for cat­a­stro­phe though).
   For myself I use the fol­low­ing.
   When I bring images in I use Light­room to import and copy all of my images to a folder on my E-Sata con­nected Drobo S (5 x 1TB hard dri­ves). My Light­room cat­a­log files also reside in one folder on the Drobo includ­ing the Light­room backup files. All files are Canon CR2 Raw.
   Nightly I have a pro­gram run­ning on my sys­tem called Best­sync 2012 that syn­chro­nizes the Light­room cat­a­logs and the image files from the Drobo fold­ers to a Medi­a­sonic Raid 1 enclo­sure con­nected via USB 3 with 2 x 2TB Dri­ves. I also have Best Sync set up to syn­chro­nize the Medi­a­sonic back­ups to one of 2 USB Exter­nal USB dri­ves when the are plugged in to the sys­tem as well as on a nightly sched­ule. Every cou­ple of weeks I swap these dri­ves and take them to my par­ents house where I con­nect to them to a Pogo­plug so I can access them across the net in case there is any need.
   The stuff I show off and want to share with fam­ily and friends I put up in gal­leries at Smug­mug (power user account). I have unlim­ited Stor­age there and they take large high qual­ity jpg files plus I like their gal­leries.
   I chose Best Sync because it wasn’t that expen­sive, Pro License was 38.00 USD and it offered VSS (Vol­ume Shadow Ser­vice) so that open files are synced in case I am still work­ing on them or left Light­room open acci­den­tally. You can set up tasks to do almost any­thing you need and have mul­ti­ple dif­fer­ent tasks and dif­fer­ent des­ti­na­tions going at dif­fer­ent sched­ules. For exam­ple I sync Music files to my home server as well as Out­look PST files and Quicken files. In addi­tion Best­Sync offers com­pres­sion, encryp­tion and a backup vault for files that get deleted on the source folder. These are backed up from the tar­get folder before being deleted there in case the dele­tion was acci­den­tal.
   I also use Win­dows Live Mesh to sync some busi­ness doc­u­ments to the cloud and some smaller stuff I use Drop­box and box.com.
   Best Sync was the way I decided to go in Octo­ber after com­ing down one morn­ing to find my Drobo report­ing as a RAW drive (talk about heart fail­ure). I used Zero Assump­tion Recov­ery to get all the data off that drive (it took about 60 hours) refor­mat­ted the Drobo and it has been solid since and even cor­rectly reported (and pro­tected data) from 2 failed hard dri­ves in a 2 week span (all from the same batch at Sea­gate). Although I am not com­pletely sold on the Drobo any more and may look for another Raid 5 solu­tion.
   Another free pro­gram you can use is Microsoft’s Sync toy. It can be sched­uled to run as a sched­uled task as well but doesn’t sup­port VSS. But it can mir­ror or sync from one drive to another.
   One addi­tional thing I was going to start doing at the cost of some addi­tional stor­agte space was to save Light­room adjust­ments as XMP side­car files. Doing this would allow them to be reim­ported into a fresh copy of Light­room with all of the adjust­ments made avail­able.
   For travel I store images on a lap­top hard drive, then to 2 exter­nal hard dri­ves and then to a hyper­drive UDMA color drive. I also try to avoid delet­ing images off of CF cards until I return home and trans­fer all of my images from my lap­top into my backup sys­tem via a Light­room Cat­a­log export from the lap­top.
   There are some great new appli­ances com­ing out in the next lit­tle while as well built on Win­dows Stor­age Server Essen­tials that will allow for NAS and client backup. I saw one from West­ern Dig­i­tal back in Octo­ber that looked very promis­ing.
   While my backup sys­tem may sound con­fus­ing it is all accom­plished by the Best Sync Soft­ware. I have found it very flex­i­ble and I can tai­lor it for all of my own uses.
   Stephen Kennedy
   Cal­gary, Ab, Canada

   Reply
   • admin says:
     March 6, 2012 at 4:56 pm

     Thanks muchly for shar­ing your work­flow Stephen!!!

     Reply
3. Ken Wolter says:
   February 24, 2012 at 9:40 pm

   Marko,

   Love your pod­cast. I hope this of use to oth­ers. I too have tried on-line backup solu­tions as men­tioned in your pod­cast and have con­cluded that it really doesn’t meet my needs. Like you I have many giga­bytes of data and it’s hardly plau­si­ble to spend months mov­ing data over my broad­band con­nec­tion to these com­pa­nies’ servers. In addi­tion, I can’t jus­tify buy­ing a stor­age appli­ance such as those made by Drobo. As a result, I have imple­mented my own solution.

   A bit of back­ground is in order. I am a PC user run­ning Win­dows 7. My work­flow is Light­room 3, var­i­ous plug-ins and Pho­to­shop. I shoot Canon, import to .DNG and out­put mostly to .JPG for web post­ing. I archive both 16-bit TIFFS and uncom­pressed .JPG’s.

   My backup solu­tion is com­prised of three exter­nal USB2 hard dri­ves. Let’s call them Drive1, Drive2, and Drive3. When imported into Light­room, pho­tos go directly to Drive 1. My Light­room cat­a­log is on my computer’s local hard drive for best per­for­mance. Drive 3 is a device I run back and forth from my office to an off-site stor­age facil­ity. If per­son didn’t have such a place, they could always take it to a friend’s house, store it at work, or even put it I sup­pose in a bank safety deposit box. I already have a stor­age locker – that’s what I use. Typ­i­cally I make a trip back and forth once a month. Peo­ple need deter­mine their own com­fort lev­els I suppose.

   I use Microsoft’s Robo­copy pro­gram. It’s free and avail­able by down­load from Microsoft. Infor­ma­tion regard­ing Robocopy’s use includ­ing its switches is chron­i­cled on the Inter­net. It looks intim­i­dat­ing at first, but I believe most peo­ple with mod­er­ate com­puter skills can adapt this to their needs. One of the key switches I use is /MIR. This allows me to “mir­ror” folder con­tents, mean­ing that if I decide to delete a data on the par­ent drive, I don’t have to do so on the two other dri­ves. The /MIR para­me­ter takes care of that for me.

   I have cre­ated Win­dows Sched­uled tasks to run the three batch files as show below. They could also be run man­u­ally but I like the idea of it hap­pen­ing auto­mat­i­cally in case I for­get. I set my tasks to exe­cute once a day. Note too that log files are stored in my case at C:\logs for review.

   1. Photocopy.bat to Copy Images from Drive 1 to Drive 2:

   C:\Windows\System32\robocopy.exe g:\photos i:\backups /MIR /xd dirs $RECYCLE.BIN /R:0 /W:0 /NP /TEE /XF *.tmp /LOG+:c:\logs\photos.txt

   (This gives me a sec­ond copy of my images)

   2. LRcatscop.bat to Copy my Light­room Cat­a­log to Drive 2:

   C:\Windows\System32\robocopy.exe C:\Users\Ken\Pictures\Lightroom\2012\2012–1\Backups i:\Backups\LRcats /MIR /xd dirs $RECYCLE.BIN /R:0 /W:0 /NP /TEE /XF *.tmp /LOG+:c:\logs\lrcats.txt

   (This gives me a sec­ond copy of my Light­room Catalogs)

   3. Offsite.bat to Copy Images from Drive 2 to Drive 3:

   C:\Windows\System32\robocopy.exe I:\backups Z:\photos /MIR /xd dirs $RECYCLE.BIN /R:0 /W:0 /NP /TEE /XF *.tmp /LOG+:c:\logs\offsite.txt

   (This gives me a third copy of my images to Drive 3)

   I like my solu­tion. It gives me a great deal of peace of mind in that images and cat­a­logs are at least two places and a third off-site when I do my monthly run. More­over, I haven’t spent an arm and a leg.

   Ken Wolter
   Still­wa­ter, Min­nesota USA

   Reply
   • admin says:
     February 27, 2012 at 9:01 am

     Thanks so much for tak­ing the time to write that. I like your solu­tion a lot and I’m doing some­thing sim­i­lar at this point.
     If any­one is read­ing this — don’t wait! get your backup strat­egy in place today!
     Thx again Ken!

     best! Marko

     Reply

Leave a Reply to admin Cancel reply