Implementing Amazon S3 for WordPress Blogs

Note: Since writing this, we’ve since migrated to WPEngine with a Content Delivery Network powered By MaxCDN, a much faster CDN than Amazon.

Unless you are on a premium, enterprise hosting platform, it’s difficult to get enterprise performance with a CMS like WordPress. Load sharing, backups, redundancy, replication, and content delivery don’t come cheap.

Many IT representatives view platforms like WordPress and use them because they’re free. Free is relative, though. Put WordPress on a typical hosting infrastructure and a couple hundred simultaneous users can bring your site to a grinding halt. To assist in my blog’s performance, this week I modified my installation of WordPress push all of the graphics from Amazon S3 (Amazon Simple Storage Service). This leaves my server to simply push HTML via PHP/MySQL.

Amazon S3 provides a simple web services interface that can be used to store and retrieve any amount of data, at any time, from anywhere on the web. It gives any developer access to the same highly scalable, reliable, fast, inexpensive data storage infrastructure that Amazon uses to run its own global network of web sites. The service aims to maximize benefits of scale and to pass those benefits on to developers.

To convert the site for Amazon S3 took a bit of work, but here are the basics:

Sign up for Amazon Web Services.
Load the Firefox Add-on for S3. This provides you a great interface for managing content in S3.
Add a bucket, in this case I added images.marketingtechblog.com.
Add a CNAME to your Domain Registrar to point a subdomain from your site to Amazon S3 for virtual hosting.
Download and install the WordPress plugin for Amazon S3.
Set your AWS Access Key ID and Secret Key and click update.
Select the subdomain/bucket you created above for the Use this bucket setting.

The next steps were the fun part! I didn’t want to just serve future content from S3, I wanted to serve all content, including ads, themes, and past media files.

I created folders for ads, themes, and uploads in my bucket on S3.
I backed up all my current content (image and media files) to the applicable folders.
I modified my CSS file in my theme to pull all images from images.marketingtechblog.com/themes.
I did a MySQL search and replace and updated every reference to media content to be displayed from the S3 subdomain.
I updated all image references for ads to be displayed from the ads folder on the S3 subdomain.

From here on out, I simply need to upload media to S3 rather than using the default image upload dialogue for WordPress. The plugin does a fantastic job in putting an S3 icon in the same location of Upload/Insert icons in the WordPress admin.

Moving all of the data and running on S3 for a couple days now has resulted in $0.12 in S3 charges, so I’m not concerned about the fees involved – perhaps a few dollars a month is what it will cost. On the plus side, if I get a ton of visitors, I should be able to handle many more than the current platform handles. My site is loading the home page in about 40% of the time that it used to, so I’m quite happy with the move!

The nicest thing about this move is that it didn’t actually require any development!

About Douglas Karr

Douglas Karr is the founder of The Marketing Technology Blog. Doug is President and CEO of DK New Media, an online marketing company specializing in inbound marketing leveraging social media, blogging, search engine optimization, pay per click and public relations. Their clients include Webtrends, ChaCha and many more. Douglas is also the author of Corporate Blogging for Dummies.

View all posts by Douglas Karr →

Anonymous

Hi,

I have an Amazon S3 account, but after trying to figure things out, I just left it because it is too difficult. Does the firefox addin for S3 make it a lot easier?

http://marketingtechblog.com Douglas Karr

Hi Ramin,

The Firefox Add-on was really a key piece of the puzzle. You need to absolutely have a bucket in place before the plugin works – so that makes it a snap.

Doug

http://CarltonBale.com/ Carlton Bale

I should add, you will need to point your CNAME to the new your_unique_cloudfront_distribution_name.cloudfront.net instead of to your_unique_subdomain.s3.amazonaws.com. But after that, you treat it just like a normal S3 bucket.

It does cost more when using the higher speed/low latency CloudFront option. If you decide you’d rather switch back to the standard S3 version, just switch your CNAME to point back to s3.amazonaws.com instead.

About a year ago, I wrote <a href="http://www.carltonbale.com/tag/amazon-s3/"a few blog posts on Amaon S3 for anyone interested.
http://CarltonBale.com/ Carlton Bale

If you're looking for even more of a speed increase, turn your Amazon S3 Bucket into a Amazon CloudFront bucket, which creates a true global multi-server, low latency Content Distribution Network. Here a link with all the details: http://aws.amazon.com/cloudfront/faqs/

Also, the wp-supercache plugin can give tremendous speed increases on high-traffic sites as it greatly reduces CPU load and database calls.

http://marketingtechblog.com Douglas Karr

Very cool, Carlton! So it's very much a distributed network such as Akamai. I didn't realize they had that available! I may take advantage after seeing some of the costs.

I have had caching with wp enabled before, but I have some dynamic content so I really struggled with it since it would sometimes cache content that I actually wanted to load real-time.

itjuju

Douglas,

From their description it sound's like Amazon is doing something completely different, they say:

“Amazon CloudFront uses 14 edge locations in major markets worldwide. Eight are in the United States (Ashburn, VA; Dallas/Fort Worth, TX; Los Angeles, CA; Miami, FL; Newark, NJ; Palo Alto, CA; Seattle, WA; St. Louis, MO). Four are in Europe (Amsterdam; Dublin; Frankfurt; London). Two are in Asia (Hong Kong, Tokyo).”

Their basically taking advantage of internet exchanges to leverage their closeness to the end user where as CDN's like Akamai have servers much closer to the end user usually within the ISP's network.

Amazons way of doing it is a lot cheaper and more effective Akamai.

Rogerio – http://www.itjuju.com/

itjuju

I wouldn't say it?s difficult to “get enterprise performance with a CMS like WordPress.”

It's all in how you setup your infrastructure or the way you host your CMS.
The way the CMS itself has been coded can also play a big part in its performance as Carlton pointed out with using the wp-supercache plugin.

It would have been better if the functionality of the wp-supercache plugin was built in to wordpress from the start – but that would require re writing the front end. Which is what lightpress.org did.

Off loading static content to something like S3 is a good way to offload processing and delivery from the main server. It?s an easy and convenient way of tapping into Amazons infrastructure to do the heavy lifting but once you reach a cretin threshold, Amazon will start to get expensive and it will be cheaper to do it in house and go with a CDN.

Rogerio – http://www.itjuju.com/

P.s.
I've been thinking about that situation for a bit, if just 100 people got together and contributed each month the price of a decent server that they would normally be paying for they could build/put together hosting infrastructure that could handle almost anything.
Bill Dawson

$0.12 for the first couple days of S3 services. Would you revisit the topic in a few months and show some stats of traffic versus costs? It'd be interesting to see how the cost breaks-down to unique visitors and against ad costs or other inputs.

http://marketingtechblog.com Douglas Karr

Absolutely! Will share monthly and annual data as soon as the bills start coming in!

Anonymous

It’s been a couple of years now… any updates?

http://www.marketingtechblog.com Douglas Karr

Yes – did you notice my opening sentence? Amazon’s S3 service was a bit slow so we moved to MaxCDN as our CDN and love it!

Anonymous

I’m sorry; that just didn’t sink in for me. Thanks.

Pingback: WordPress iPhone Plugins: Admin and Theme | The Marketing Technology Blog
Scout

If you are Windows you can use S3 Browser – http://s3browser.com to upload files, such as images, scripts, etc. to Amazon S3 . Must have tool.

And thanks for the useful post!
http://www.optizmo.net Grant

Amazon S3 is an incredibly well valued service. I'm just in the process of integrating it into a CMS. The only issue that I've come across from a development perspective, not an Amazon service perspective, is that if you want your user to transparently upload the file directly to S3 via POST and you have a multipart form that incorporates text destined for your local database, you're stuck. You either need to separate it into two forms, or try use ajax to upload the file first then on success submit the data locally.

If anyone has a better solution, feel free to let me know )

Nonetheless, the cost savings for hosting large high traffic files warrants the development of such a system.

Grant

Suppression List Management Systems
Pingback: Why Hasn’t Your Company Implemented a CMS?
Pingback: We Moved WordPress to MediaTemple
Pingback: 5 Easy Ways to Speed Up Your WordPress Site
Scott Judson

Hi,

Great write up. I have stepped through as you describe, but in my admin panel where I upload images, I dont see an S3 button. I have noticed that my images, when uploaded normally end up on Amazon, does this mean I can now copy all my existing images over and delete the ones on the server?

And do I need to modify where my images come from or does the plugin do this?
Douglas Karr

Hi Scott,

You should see a little database looking icon to the right of your typical icon. That’s the icon to pop up the Amazon window. I moved all of wp-content/uploads to Amazon and made sure I had the same path… the only difference being the subdomain. They were at http://www... and now they’re at images.marketingtechblog.com. After I copied all the images over to Amazon, I used PHPMyAdmin and did a search and replace for src=”http://www.marketingtechblog.com and replaced it with src=”images.marketingtechblog.com. (http://www.marketingtechblog.com/wordpress/mysql-search-replace/)

Hope that helps! It’s not seamless, but it works.

Doug
Scott Judson

Hey Douglas, thanks for that, I have updated the DB so all images point to images., but I see some of the thumbs (when looked at via the page info) shows the iamge still at www.

Here is the site (www.gamefreaks.co.nz) – a, alos having some major memory issue for the front page, only started once we shifted hosting, hence me now looking at offloading some of the hosting pressure to S3.
Scott Judson

do you know if it is possible to just upload to S3 without it also storing a copy on the local server?

Tnx
Douglas Karr

Hi Scott, the S3 plugin provides its own path directly to Amazon, so the file is NOT stored locally.
Simplyshredded1

Hi Joe,

Great Post!

Does this wordpress plugin “the one you mentioned”

http://tantannoodles.com/toolkit/wordpress-s3/

Work with the latest version of wordpress?

Would be very interested to know if it is compatible, because I dont think it has been updated in a while. Appreciate the help

http://www.marketingtechblog.com Douglas Karr

It is compatible with the latest version, but I honestly don’t like the way it works – you have to shift and load all images to S3 with a different process. We may actually build a more robust CDN (Content Delivery Network) integration with WP that synchronizes rather than demands a different process.

http://twitter.com/diptoe Edgar Diaz

Great post! I’m somewhat new to this, and didn’t realize you could offload media. It makes sense though. Thanks for sharing!
Sander

Do you know if this works with “External Buckets” too? I want to set this up for a blog of a friend of mine and let him use a bucket in my AWS account (I already created a user account for him and gave him access to one of my buckets using the Amazon IAM tools).
Anonymous

Great article! Thanks for posting. Helped me get setup and running.
Pingback: Speed Up WordPress with Amazon and W3 Total Cache | Marketing Technology Blog