6 Years Worth Of Photos…Gone
I heard a lot of people griping about how bad 2016 was for them. While many people had one or two really bad things happen…I had several. Although I don’t want to go into all the personal details of my life right now, one of these terrible things to hit me was 6 years worth of my work vanished. I want to share my experience on this to try and help others before it happens to them…or help others who might experience something similar.
For starters…what happened?
Several year ago I entered a contest to win a storage device called a “Drobo 5D”. To my surprise, I won. I was extremely excited about it! These devices run between several hundred dollars to several thousand. Since I was able to save money on the device, I dropped a bunch of money on 3tb western digital red edition hard drives to fill the device up and get started.
About a year after having the device, it started acting up. Luckily I had been using Backblaze (an online data backup company) to back up my data. At this point in time I had several terabytes of data stored but it was still well under capacity. Out of no where the Drobo died and I lost all of my data. I was pissed. I had read reviews from Scott Kelby and others saying they didn’t like the Drobo and knew I was possibly getting in over my head with it. I was lucky though, my data had been backed up. I bought a couple external drives and downloaded my backups from Backblaze.
A little while had passed. I was a subscriber to the Backblaze newsletters and was always reading the latest info about hard drive statistics. Turns out the Western Digital 3tb red drives were unreliable as hell. I thought to myself, maybe I should give the Drobo a second chance. After all, I am sick of seeing my data spread out across 5 hard drives and not being able to free up any of my USB ports on my iMac.
I bought a pair of Western Digital 6tb Green drives, opened up my Drobo and started transferring my data back to the Drobo. Little by little I would purchase new drives to put into the Drobo as I would continue to fill it up. Magically, it worked!
For a few years now I have been running without any hiccups. It seemed too good to be true…until 2016 struck. My marriage was in shambles, I needed to move out and found a place to move into. I found a decent place in the city. It was old but it would work for me. Within a few weeks, more devastating news just kept pouring in. I was feeling completely drained. All I could do to keep my spirits up were photo-shoots. My Drobo started acting up again. I thought “oh-no”! I had been here before so I wasn’t too afraid. Plus, I had been backing up my data in the background for years now at this point.
I did a little investigating and discovered I was at about 23% left of capacity of the 17tb maximum allowed by the Drobo Software. You see, although I had more hard drive space available than 17tb, the Drobo itself does not allow for more data than 17tb per partition. So I had it setup to have a secondary partition which I called “Drobo Extended”. Now the interesting thing here is that the software believes that each side has 17tb available. In reality, at max capacity I had 22tb of hard drives ((5)-6tb drives+(1)4tb drive)-(1)6tb drive for drive redundancy=22tb. So I knew I couldn’t just willy nilly take all of my data from one drive and transfer it all to the other side. It wouldn’t all fit. So I would transfer a bit here and then delete it from the originating side. I did this little by little because I knew I would need to transfer everything at once in Lightroom to keep the data in tact.
For some reason though, my data structure wigged out. The Drobo wasn’t staying connected and eventually appeared as though it was needing to be formatted to even work. I couldn’t format it yet…half of my data was still on that side of the partition. I found out that DiskWarrior has the ability to rebuild the data layout so that your files all appear in the same place as they used to be. So hallelujah, I fixed it…temporarily.
The Night From Hell
I had a really fun photo-shoot planned for the evening. We were shooting fashion in an old art studio. I had really wanted to shoot at a nearby scary motel…so everyone agreed to go to the location. Within 5 minutes of shooting, the girls had been cat-called several times, had a scary guy keep walking by and finally a man from the motel came walking over and said “you guys need to leave, dressed like that you will end up raped and murdered around here” (literally all within 5 minutes). So we quickly packed up and left. I took the girls back to their cars at the art studio and then I headed home. Since I had to pass the motel to get home I noticed every block around it was blocked off by cops and police choppers were flying over-head scoping out the area. I live only a few blocks from this location…I thought great, we got lucky leaving when we did, but some murderer is now hiding in my neighborhood.
When I got home, I checked to make sure no one was hiding in my house and locked all the doors (like a true manly man….err I mean scaredy cat). The helicopter kept buzzing my house which was making me nervous. I sat down at my computer to start importing the pics. I looked at my data and saw that it was still mid-transfer of a large group of files. I tried to import my new photos into Lightroom. Just as the police helicopter flew over-head, my power was cut. I sat there looking at a dead computer screen in complete darkness. I didn’t know what I was more afraid of, the murderer in the neighborhood or the fact that I might have just fried my Drobo. When the power came back on and my Drobo fired back-up. I tried to see the status of everything…and the power went off again. I disconnected everything and waited a while to see if the power would just stay on for a while before trying to boot all the way up. The power continued to cycle several more times that night all while watching the helicopter fly over-head over and over again. I patiently waited and when the power finally stayed on for a while, I booted up the computer.
The Drobo That Wasn’t
When everything came back up, I used Disk Warrior to again rebuild my data structure but when it finished my Drobo only had 3tb of data on it. To my horror, most of my files were gone. Everything I had shot between 2010-2016 was missing. Not to mention all of our family photos, all of my accounting, all of my music, movies…you name it, gone.
The Backup That Didn’t
All of that said, I wasn’t afraid, I had backed all of my data up over the years to Backblaze. Well apparently, they had been sending me my weekly automated emails that showed my data hadn’t moved in months…years maybe. For some reason, my computer decided to stop sending files a while ago. The way Backblaze works, currently infuriates me. When I needed my data years ago, they saved my ass. So I cant complain too much. The way Backblaze has always worked is that it saves up to 4 weeks worth of data versions. Anything before that is stored in a “Snapshot”. you must have a “completed-state” in order to have a snapshot created. Since my data has been a moving target (always growing) for several years now, it has never been able to be called a “snapshot”. I was feeling beside myself. Everything is gone. Since I had been moving my data around on my Drobo for more than 4 weeks, whole sections are missing from my backup states. 🙁
Currently this is where I am at in the process. During my time with my Drobo I had encountered several articles talking about how Drobo uses its own proprietary software making it impossible to recover your data should you ever lose it. Drobo created a platform called “beyond-raid”. I wont go in to depths here of what a raid structure is but basically its where you take a bunch of disks and the way you daisy chain them together can result in different setups ranging from mirroring each other to just looking like a bunch of disk drives. The Drobo is meant to look like one giant drive…or in my case two. From what I have discovered, it hasn’t mattered much because the software recovery tools are in fact finding my data. The bad news is, so far all of it is corrupt and unusable.
What I have learned
Since I currently do not have any answers on whether or not I will ever see my data again I will close by saying what I wish I had done all along.
Firstly, always assume the worst. Have backup in place. I would recommend a 2 prong approach to this. If you can afford it, buy (2) 2 terabyte portable drives. The reason for this is to have one drive be your working drive. The other to be your “time machine” (on Mac) or a mirror backup if on a PC. I always try to bring my recent work with me on shoots. That way I can either be editing in my spare time or show off recent work (that I can legally show) to potential clients. So it is important to me to have portability. But it is also important to know it will work…remember how I said 3tb drives were bad? Turns out 2tb drives are some of the most solid drives on the market. Because its only 2tb, it will also backup to a backup service quicker. Which brings me to my next point. GET BACKUP!
The second prong would be making sure your data is backed up. I still highly recommend Backblaze. Even though I had a hiccup with them, it was my fault for not paying attention. Because of this, I jumped ship and started using a backup company called “Crash Plan”. My reason for this jump was because there isn’t a “snapshot” involved with them. Your data is backed up with them until you delete it yourself.
Thirdly, this brings me to my next point….obviously 2tb isn’t even close to enough storage for all of my files. There are several ways to go about fixing this problem. If you have a decent amount of money, I would recommend the G Speed Studio setups from G-Technology. My sole reasoning for this is because they use HGST drives which seem to be the sturdiest longest lasting drives that money can buy. You can also buy them with a shuttle drive which is a single drive that you can take out of the tower and use the disk as needed and them return it to the tower when you are done. As you already know I love this approach but am still trying to learn if it is possible to keep that data separate so it backs up to the cloud faster….also I would want that secondary drive on some sort of mirror or time machine. hmmmm
Since I am not loaded down with cash at the moment, I settled on a WD My Book 16tb. It has 2-8tb drives inside. I am currently rescuing my data as I can and loading it onto this drive. My plan is still to get the G-Speed Shuttle setup.
Again I am going to talk about backup. First of all, you’ll notice I never mention keeping files on my main computer hard drive. I trust this hard drive to carry the OS and Applications only as much as possible. Crash Plan and Backblaze are both apps that run off of your main hard drive. Those apps will always try to backup your main drive first, so keep everything off of it so that this goes quick. Next you will want your most recent files backed up first. The reason for this is because your clients are still waiting for those files (is my guess). So my plan is to only store a years worth of files on the smaller portable hard drive. Allow your backup service to back these files up as much as possible before selecting your larger archive-storage.
Since my work that is from a year ago or older is sitting on my archive drives, this will take a massive amount of time to backup to the cloud. If you have specific files that need to be backed up first, I would recommend selecting those files in the software before just telling it to backup the whole drive. It is important to note that these backup software programs run in the background only when your computer is on/awake. So always be checking the status of your backups. If your current work is backed up now in two locations…I can say you are ready to add the archive drive. Keep in mind, if you decide to use Backblaze, an ideal way to do it would be in stages, only selecting files as a time so that you know you have a completed backup “snapshot” in between adding new files…even though they are old files to you, they are new to the backup.
Lastly, there are several other methods out there…but this one is an interesting one. I used to work at a video production company where all of the videos were shot on Beta tapes and then stored on a shelf for archival. After cameras went digital, we started charging the client for the flash cards instead of charging for the tapes. We kept a binder with business card inserts as holders for the sd cards and kept them numbered and cataloged so that we always knew which card was from which shoot. Although I personally have never done this method, it has made me think (and wish) that I had been doing this all along.
In closing, never underestimate or under-plan. It could have been a thief that stole my computer or a fire that burned my house down… instead of a power outage.