Amazon S3 and Subdirectories
I recently got a question from someone regarding how one can make subdirectories in Amazon S3. Amazon has only one level of directories, called “buckets”. There are no subdirectories, and you cannot put buckets in buckets. However, you can fake subdirectories by cleverly naming your resources.
For instance, say you have claimed a bucket named “mybucket”. Everything you place in the bucket would be accessible from the URL, http://s3.amazonaws.com/mybucket. Now, if you wanted to place an image resource deeply nested into a heirarchy, you couldn’t. But, if you name your resource path/to/my/resource.jpg, you can access it from http://s3.amazonaws.com/mybucket/path/to/my/resource.jpg. Voila!
Recently Steamed:
- 07/23/2008: Mochi Media Site Relaunch
- 07/22/2008: Amazon S3 Downtime Visualized
- 07/21/2008: NPR Launches Open API, offers Widgets
I considered this approach at first, but it has one major problem. What if you want to try and change (rename?) a folder high in a hierarchy, s3 doesn’t provide a way to ‘update’ names or files so you have to upload all the files again. I went with the system of naming assets with an unique ID and storing a virtual folder system in a database.
Comment by Alex MacCaw — 11/30/2006 @ 11:19 am
That’s a good point, but how often does one make wholesale changes to their directory structure? In the era of permalinking, this should be avoided.
This system also has the advantage of being able to host pages that have a complex hierarchy of elements. For instance, you could hose a web gallery that has images in subfolders. HTML and SWF files will still work as though they were accessing a real filesystem.
Comment by geoff — 11/30/2006 @ 12:02 pm
Why didn’t you post this YESTERDAY?!
I spent at least an hour today trying to rectify the discrepency between seeing people use and talk about subfolders in buckets but having absolutely no support for it in the api.
Comment by masukomi — 11/30/2006 @ 11:56 pm