How to store photos in the cloud and avoid vendor lock-in
I’ve been burned by relying 100% on a cloud service before. Some time ago a photographer friend convinced me to sign up for an awesome photo storage service called Everpix.
My internet connection ran red hot for a couple of days and nights until my whole photo library was finally transferred to Everpix’s cloud. And I loved the service. It was fast and had great UX. Finally, a cloud photo storage solution that worked well for me.
It was simply a joy to use.
At least for a few days—then they sent me an email telling me they ran out of money and had to shut down the company.
There was a grace period where Everpix let you download your photos in their original quality as a giant zip archive. So at least people didn’t lose any data if they acted quickly enough.
A few weeks later Everpix was finally gone and I felt frustrated1.
I had really enjoyed being able to access all of my photos from any device I owned. I had liked the fact that I didn’t have to worry about manual backups as much.
I decided that I wasn’t going to be tied to a single cloud service ever again and set out to build my own photo storage solution. It’s not as fancy as Everpix was but it get’s the job done and feels much more future proof.
Let me give you a quick overview of how it works.
One folder structure to rule them all
Instead of using a proprietary storage format like Apple’s Photos.app or Everpix, all of my photos simply go into a nested folder structure based on their timestamp.
I give each photo a path and filename based on the time it was taken and then I sort it into the following folder structure:
├── 2014 │ ├── 2014-01 │ │ ├── 2014-01-05 13.24.45.jpg │ │ ├── 2014-01-05 21.28.48.jpg │ │ ├── 2014-01-05 21.28.48-1.jpg │ │ ├── 2014-01-06 21.14.38.jpg │ ├── 2014-02 │ | ├─ ... │ ├── ... │ └── 2014-12 ├── 2015 │ ├── 2015-01 │ ├── 2015-02 │ ├── ... │ └── 2015-12 ├── ...
This is a dead simple scheme that I’ll be able to keep using as long as there are hierarchical file systems. And the good news is that all of this sorting and structuring can happen automatically based on EXIF timestamps or file creation dates.
I found that a simple folder structure is a perfect fit for my photo storage needs. I sometimes create “albums” by moving some photos into a separate folder, for example:
├── 2015 │ ├── 2015-01 │ ├── 2015-02 │ ├── 2015-02 My Album │ ├── 2015-03 ├── ...
This let’s me keep the year-month sort order in the yearly folders and provides enough structure to find important events quickly. Occasionally I also create “virtual” albums in Carousel to share with friends and family, but more on that in a minute.
Dropbox & Photosorter
The setup I use now is built around Dropbox for cloud storage and my open-source photosorter tool. The complete workflow is fully automated and looks like this:
- New photos go into the Camera Uploads folder on my Dropbox. This either happens by me manually copying them off an SD card into the Camera Uploads folder or the Dropbox iOS app automatically uploads new photos when my phone has a Wi-Fi connection.
- Photosorter runs on my home server and watches Camera Uploads for new photos. It then takes them and moves them into the appropriate place in my Photos folder which also lives in my Dropbox. Photosorter detects and ignores duplicates through their SHA1 hash. Photos taken in the same instant are deduplicated by adding a suffix (
-1,-2, etc) to the filename. - Dropbox picks up the new files in my Photos folder and distributes them to all my devices. Once the photos are in Dropbox I can also access them from anywhere using the Dropbox website.
This setup has the nice side-effect that I have a physical backup of my photos in several places, like my home server and my Mac. This works because my photo library is only about 100 GB in size. For a larger library I’m either going to just buy more storage or keep a full backup on my home server and disable syncing on my Mac.
If want to give photosorter a try there’s documentation and a deployment example on its GitHub page.
Carousel
Update: Dropbox is going to kill Carousel, meh.
Since I wrote this article Dropbox announced that they will shut down Carousel on March 31st, 2016. This is a bit of a bummer because parts of the workflow I’m describing here worked really well with Carousel.
However, they said that they’ll port most of Carousel’s functionality back into the Dropbox app and website. That’s fine by me and would work well with my photos workflow. I don’t really care which app I need to launch to look at my photos (that’s also kind of the whole point of this article). Once Carousel’s gone I’ll update the article with new recommendations for tools.
Carousel is Dropbox’s new product for managing photos in your Dropbox account. I really like the Carousel app and website. It’s a super convenient way to browse through my photos from anywhere I want. I also frequently use it to share photos with friends and family by creating ad-hoc albums on Carousel.
Their iOS app let’s me access all of my photos while not taking up much space on my phone. This is thanks to Carousel’s smart caching system that only keeps high-quality version of photos you viewed recently on your phone. It’s similar to iCloud photos on iOS 9, works well and usually requires zero babysitting.
Carousel also has a cool flashbacks feature that shows you photos that you took in the same week one or more years ago. Everpix had that too and it’s a neat way to enjoy older photos from my library.
Like I said before I also use the iOS app to automatically upload new photos from my iPhone when I’m on Wi-Fi. This pretty much guarantees that I won’t lose photos while I’m traveling. It also helps keep enough free space on my phone so I can continue taking photos.
The future
I’m currently running photosorter on my home server. At some point I might replace it with a virtual machine on S3 or Digital Ocean which will provide cheaper storage and better fault tolerance. I don’t really trust that little Toshiba notebook drive spinning 24/7.
This setup has served me well over the past two years. Obviously setting this up is more involved than just using a turnkey solution. But I also feel like it’s more future proof than using an off-the-shelf service like Apple’s iCloud Photo Library or Google Photos.
I’ve been burned by Everpix’s sudden disappearance and if Dropbox goes away I’ll just use a different filesystem-based sync service like BitTorrent Sync. If you’re worried about privacy then running your own photo storage solution might be appealing, too.
