Amazon, long known for their amazing on-line book store, had in the process of setting up the companies backed infrastructure, ended up with some amazing services, that they decided to release upon the general public in January of 2006. Suddenly Software as a Service (SAAS), was a reality for the masses, available to anyone who wanted to piggy back on the services Amazon released. Probably the service that caused the biggest buzz was Simple Storage Service (S3). For all intents and purposes this is a glorified file system, with some interesting bells and whistles. However the biggest benefit to most, is probably going to be the speed of delivery and also the reliability, that the might of Amazon can provide. That being said, this is in no way a perfect service. Most hosting services, usually boast an uptime of 99.9%. I’m not sure on S3’s uptime, but I’m pretty sure it’s in the high 90’s, but they are not without their share of issues. A couple of times in the last year, they have had fairly large outages, in excess of half a day, but of late, things have been calm and reliable. They do provide account credit should their level of service drop below certain levels, which you can read about in their Service Level Agreement (SLA).

While S3 is truly an amazing service, it’s not right for everyone. While their basic charge of 15 cents per gigabyte seems fairly cheap, I looked at using for my off-site backups and it wasn’t until I did the calculations on the Amazon S3 Calculator page.

I have about 200 gigabytes of data that I need to backup off-site, and when I plugged in the numbers into the S3 calculator, it came back with a figure in excess of $30 per month and that didn’t include all the video stuff I have. If you are using S3 in conjunction with a business idea, this will probably not be an issue. Indeed there are manyh companies that publicly state they use S3 as the backend to service their files. Two of the popular ones that come to mind are Smug Mug and Animoto, who use S3 to server photos and videos. When you consider the sheer weight of traffic served, they are finding S3 a very cost effective solution.

So where do we start?

1. Setting up an Amazon S3 account
Setting up an Amazon S3 account is a bit of a slog, but for anyone who already has an Amazon account, it’s at least slightly easier. First things first, you’ll need to go to the Amazon Web Services web page, http://aws.amazon.com/. Once the page has loaded, click on the [Sign Up Now] button:

If you already have an Amazon account, this would be the account through which you order books, you can log on, otherwise you’ll need to follow the steps, starting with the following screen:

If you have an account, login using your email and password, otherwise, enter you email address and then ensure “I am a new user” us selected, then click on the [Sign in using our secure server] button. You will now be presented with the following screen:

Complete this with your details and then click on the [Continue] button. Sign up is nearly complete, for the main Amazon part anyway. The following screen also needs to be completed with your personal details, address, etc. You will need to agree to the Amazon Web Sevices Customer Agreement, to continue:

Now those details are filled in correctly, click on the [Continue] button. An email will now have been sent to the email address you specified. One you have received the confirmation email, your Amazon account is set up. You now need to set up Access Key, to be used in the Simple Storage Service (S3) service. In the confirmation email, click on the link, relating to Setting Up an Access Id. On the page that is displayed, are two key pieces of information you will need for a variety of services and software you will use. Keep a note of them here for safe keeping. You will also probably want to keep an electronic copy, as it is much easier and less error prone, to copy and paste these keys, rather than typing them out. Please store them securely. The key screen will look something like this:

Your Access Key ID, will obviously not be obscured, but you will need to click on the [+ Show] to see your Secret Access Key.

Your Access Key Id : _______________________________________________________________

Your Secret Access Key : ____________________________________________________________

Now click on “Products” at the top of the screen and then click on the Amazon Simple Storage Service (Amazon S3) link. You will probably want to explore a few of the links on this page. When you are ready, click on the [Sign Up for Amazon S3] button. You are now ready to link your credit card to your Amazon account. This is the credit card to which the monthly charges will be charged. Once you have verified your credit card details are correct, click on the [Continue] button.

Click on the [Use this address] button if the billing address is the same as the address you entered earlier, otherwise enter a new billing address and then click on the [Continue] button.

You know have the opportunity to verify that everything you entered is correct. Once you are satisfied, click on the [Complete Sign Up] button. You will now receive a confirmation email. Sign up is now complete.

2. Uploading files
Although Amazon do not provide any means to manage the files on your S3 account, there are numerous applications and websites that fill in this vital missing link. One of the most popular is the S3Fox add-on for Mozilla’s firefox browser. If you have not used Firefox before, it is an excellent, extensible browser, with many neat add-ons. It is also considered one of the safest browsers, with many built in protections against pop-up and phishing schemes.

If you haven’t already installed Firefox, you can get it from the following webpage:
http://www.mozilla.com

Once you have installed Firefox, you need to install the S3Fox add-on. go to the following web page:
http://www.rjonna.com/ext/s3fox.php

Now click on the “download it here” link, which you can see here:

As part of Firefox’s security, it will not allow you to install this by default. You will see the following appear at the top of the browser window.

Click on the [Allow] button to begin the install of S3Fox. You will now see the following screen.

The [Install Now] button will be active after 3-5 seconds. Click on [Install Now] when it is ready. When the installation is complete, you will be prompted to restart Firefox. To do this, click on the [Restart Firefox] button.

Now that S3Fox is now installed, you will find the S3Fox icon at the bottom right of the status bar:

You can start S3Fox by clicking on this logo. The first time you start S3Fox, you will get the following message:

This is basically informing you that S3Fox needs some key information before you can proceed. Once you click the [OK] button, you will see the main screen, which looks like this:

To set up your account, click on the [Manage Accounts] button and the following screen will appear:

Enter anything you want to describe your account. Then copy and paste your Access Key and Secret Key into the text boxes. Please ensure that no extra spaces are added to the end of your keys when you paste. When you are satisfied that you have entered everything correctly, click on the [Close Button]. Then click on the minimise button next to the [Manage Accounts] button. S3Fox is now set up. This screen can be used for data transfers, by dragging files to the remote view, however a more conventional screen is also available. Click on the “Tools” menu item in the main Firefox menu, and then click on the “S3 Organizer” menu item:

A two pane view is now displayed, showing your local hard disk and your S3 space. You can create buckets (directories), and copy files two and from both views.

3. Making files available to the public
By default all files and directories created, are only visible to you. Even if you know the URL to get to the file, you will only get a pretty terse XML file :

This means that you can store file securely, and at the same time have a folder that has files available to the public. It’s even quite feasible to have both public and private files available in the same folder, but this could potentially get quite confusing. It make be a good idea to create both a public and private folder (bucket). Remember to bare in mind that all folder names have to be unique across the whole of the Amazon S3 service, so you may want to having a naming convention, something like name-video-public and name-video-private.

After you have uploaded a file to one of your folders, that you wish to be made available to the public, the next step is to change the files rights, using Amazons Access Control List (ACL). This is similar to changing a files rights within Windows, Linux and a whole host of other operating systems. You’re basically telling the Amazon S3 service, who can do what with the files you upload.

With the S3Fox Organizer open, ensure you have created at least one folder (bucket) and then upload a file to it. When the upload has completed, open the folder by double clicking on it, if you haven’t already and you will see the file you just uploaded. Now right click on the file and select “Edit ACL” from the menu that appears. You will now see the following screen:

To now make this file publically available, click on the red “X” next to Everyone and the click on the [Save] button. This file is now available for anyone to download, that is once they know the URL. You can easily get the files URL, by again right clicking on the file and selecting “Copy URL to Clipboard”. If you now go into your browser and in the address bar, paste the URL you have on the clipboard, when you press enter, you should now see the file you uploaded, if it is an image, or see the usual download dialog.

As I mentioned before if you want to make all files in a specific folder, public, you can right click on the folder name, and again select “Edit ACL” and repeat the process we did for a file. Now when all files are uploaded to this folder, they will be publically available.

4. Hosting a Complete Site on Amazon S3
While it is quite possible to host your website on Amazon S3, this is only feasible if you site consists of static pages, i.e. pages that are not dependent on some kind of backed programming language, like PHP, Ruby, Python, ASP.NET and Coldfusion, to name some of the most popular. This is because the scripting language your site is dependent will not be available on Amazon S3. That’s not to say you cannot host interactive sites, just that your choice of languages will need to be client based, i.e. Javascript and Flash (although even then some Flash options will not be available). Remember S3 is geared primarily at providing a means of serving up large files, quickly and realiably.

5. Making Content Look Like it is Being Served from your Website
As you will have found out, the URL’s S3 generates for your files is pretty long, take this one for example:

http://colboys-videos.s3.amazonaws.com/cdm.jpg

A much more attractive alternative would be:

http://videos.colinmeeks.com/cdm.jpg

This makes it look like the file is coming directly from your website, but more importantly, it means should you decide to cease to host your files with Amazon, you just need to create a sub domain that replicates the one you create for S3 and there will be no disruption in the files you are serving, which is very important should you care about links being picked up by search engines remaining valid. If you decide to go down this route, you’ll probably want to keep a duplicate of the content you upload to S3 on your own website.

To set this up is relatively straightforward, but is not covered in this document.

6. Final Thoughts
S3Fox is just one of a many of programs that helps with managing your S3 account. It’s also free, but not only that, it’s very well written. However should you prefer to have something that doesn’t run in a browser, there are a few programs available. One of the most popular is JungleDisk, this is an excellent program that has a myriad of features, the most important one, for me anyway, is the ability to map a folder to a drive. That is, I can have my folder “Backup” mapped to the logical X: drive on my computer, and copy files to and from the X: drive. This is very convenient, but really comes into it’s own when you have JungleDisk installed on multiple computers, meaning the X: drive is common to all drives. I use this a lot for backups, but also as a central place to hold written works in progress, meaning I can work on documents on which ever computer, without needing to email it myself or carry it around on a flash disk drive. The only downside to JungleDisk, is that it uses it’s own method to store the files, so it is not able to use this to make content publically available. JungleDisk costs $20, and you are allowed to install on as many machines you want that share the same S3 account.

The second program is called BucketExplorer. This is the only client I’ve find that allows you to set up the more extensive permissions on files, including time limited URL’s. I haven’t yet used this personally, but it looks to be pretty good value at $49.95.

Amazon is never one to rest on it’s laurels and there are a host of other features sure to be coming down the pipes. Suffice to say, S3 has landed with a a sizable thump, providing an excellent service with Amazon’s weight behind it.

convert this post to pdf.

Filed under: Unpublished Article