FWIW: Amazon's Micro instances will burst to saturate unused CPU cycles on the host machine up to a hard-capped limit before being choked to death by the hypervisor.
Had he run the Blitz test for 10 mins or more you would see the spike in traffic up, beyond what you think a Micro should sustain, and then it plummet to near 0 for a disturbingly long period of time[1].
If you are unlucky enough to have a Micro on a host that is fairly saturated, the performance you get is untenable.
Micro's are not "smaller than" Smalls -- they are a different type of monetization production from AWS allowing you to pay cheaply for little bursts of underutilized hardware.
There is no way (none... not possible, zero) that a Micro would be able to provide the bandwidth and CPU power to host a realistic site doing 10 million hits a day even if everything was straight from RAM.
Read through the EC2 forums for any length of time and you'll frequently find people coming in with reports of their machines "stopping" or "crashing, and totally inaccessible" for minutes at a time with a 100% ST ratio; every time it is a Micro that has been hammer for a bit either through benchmarking or use before the hypervisor puts it in a full nelson and brings it down to the point that SSH connections to the host cannot even be maintained.
-- I would also point out that not only is the CPU time allocated in bursts for a Micro, but the bandwidth is prioritized behind every other instance type (unless you were using a CDN, this would make hosting a typical wordpress site sluggish at best from a Micro -- and again, doing 10 mil hits a day... not going to happen in reality) -- yes you can offload all your graphical assets to a CDN, but now this article is about a $15/mo server and a $4217/mo CDN bill which is a very different article.
Additionally, if you need to use EBS at all in here, the story gets even worse with Micro's (even using something like RDS which requires network I/O in addition to hosting site content is all going to collapse on itself within the first 5 mins of the site's life with traffic like that).
All that said, the tips in the article are great. I only mean to clarify expectations with the use of Micro's. A whole swarm of them grinding through a work queue in random order is great; using them as the backbone of your web presence will have pain points. (Yes you can put 20-50 of them behind an ELB, but at that point why not run a handful of Mediums or a few Larges).
Anyone with a Blitz.io account, please feel free to setup this exact configuration and run the benchmark for 1hr with the same load to verify the meltdown.
[UPDATE] Ewan, I am not knocking your article, I wouldn't expect most people to be familiar with the ins and outs of every cloud provider. The tips and techniques are great regardless of the actual performance on a Micro (and applies to anyone trying to scale WordPress). Just wanted to clarify for anyone getting excited that they can now run their Fortune 500 website on a single AWS Micro that there are nasty surprises lurking just under the surface.
I only picked the micro because it's cheap, and AWS let me fire one up, build the config, break it, trash it, and start again, all in rapid succession.
One thing though, this configuration doesn't actually need much real CPU or disk resources - it's pretty much all memory, and as far as I know, AWS doesn't overcommit RAM. this means it should be "relatively" stable. The CPU usage is at around 5-10% even at the peak.
Personally, my own blog runs on Linode, because I think EBS is broken, but each to their own :)
+1 right you are about micro instances getting hammered by the hypervisor if you use too much sustained resources.
One trick that I used when I was temporarily hosting a small Clojure web app for a customer: I ran the web app using "nice" to reduce its priority. I did not do any measurements, so this is really subjective, but the app seemed to run a lot better as far as consistently getting a little bit of processor time to run.
Not so subjective: I did some Clojure development in a repl on a micro instance (because it was already set up for access to a Hbase cluster) and doing a "nice lein repl" really made development possible, if a little slow.
After reading this my question is: will his system work on a small instance on ec2 instead of micro? It sounds as though the majority of issues you're describing are micro-specific and while there's a thousand "tune wordpress" blog posts out there it sounds like this one is a good idea minus the use of a micro instance.
The techniques Ewan outlines are spot-on; my clarification is for people to have the right expectations of the Micro's, not on the information he provides on tuning a WordPress install.
That said, Smalls and up on EC2 will give you more consistent performance (they aren't built to provide the spike-performance Micros are) BUT are still relatively subject to "noisy neighbors".
The smaller the instance, the bigger the impact felt; it is still possible to have a small grind to a halt because of a noisy neighbor and typically I do not see people hosting web front ends with smalls because of the erratic and poor performance (in general). Possibly a fleet of them behind an ELB, but even then I see Mediums and Larges much more often as the "web server".
The mediums provide an awesome (and typically unsung) balance between monthly cost and performance on your way to a large (which can get quite expensive).
Great makes a lot of sense thanks for expanding. Like you touch on I think price is the big issue for most people who aren't looking to build for 15mm/month but are just looking to sleep at night knowing that getting to the frontpage of digg, reddit, techcrunch, hackernews, etc won't sink the ship.
I think "a $4217/mo CDN bill" may be a bit of an exaggeration :p I honestly had no idea how much CDN's charge, threw some rough stats into AWS Monthly calculator for CloudFront (596GB/Month, 500kb avg. object size) and the bill was only $73.87 /month. Call me surprised!
Sounds about right to me. How did you get to 596GB? That seems very low for 300,000,000 visits (remember this was a daily figure, but you're paying by the month). If you have 10 requests per visit (reasonable for the low end) that turns into a cool 3B requests. That itself is $2,250 a month.
A million visits times 500KB = 476.84GB. Per day, so 14TB per month. At best that's another $1,600 a month.
Had he run the Blitz test for 10 mins or more you would see the spike in traffic up, beyond what you think a Micro should sustain, and then it plummet to near 0 for a disturbingly long period of time[1].
If you are unlucky enough to have a Micro on a host that is fairly saturated, the performance you get is untenable.
Micro's are not "smaller than" Smalls -- they are a different type of monetization production from AWS allowing you to pay cheaply for little bursts of underutilized hardware.
There is no way (none... not possible, zero) that a Micro would be able to provide the bandwidth and CPU power to host a realistic site doing 10 million hits a day even if everything was straight from RAM.
Read through the EC2 forums for any length of time and you'll frequently find people coming in with reports of their machines "stopping" or "crashing, and totally inaccessible" for minutes at a time with a 100% ST ratio; every time it is a Micro that has been hammer for a bit either through benchmarking or use before the hypervisor puts it in a full nelson and brings it down to the point that SSH connections to the host cannot even be maintained.
-- I would also point out that not only is the CPU time allocated in bursts for a Micro, but the bandwidth is prioritized behind every other instance type (unless you were using a CDN, this would make hosting a typical wordpress site sluggish at best from a Micro -- and again, doing 10 mil hits a day... not going to happen in reality) -- yes you can offload all your graphical assets to a CDN, but now this article is about a $15/mo server and a $4217/mo CDN bill which is a very different article.
Additionally, if you need to use EBS at all in here, the story gets even worse with Micro's (even using something like RDS which requires network I/O in addition to hosting site content is all going to collapse on itself within the first 5 mins of the site's life with traffic like that).
All that said, the tips in the article are great. I only mean to clarify expectations with the use of Micro's. A whole swarm of them grinding through a work queue in random order is great; using them as the backbone of your web presence will have pain points. (Yes you can put 20-50 of them behind an ELB, but at that point why not run a handful of Mediums or a few Larges).
Anyone with a Blitz.io account, please feel free to setup this exact configuration and run the benchmark for 1hr with the same load to verify the meltdown.
[UPDATE] Ewan, I am not knocking your article, I wouldn't expect most people to be familiar with the ins and outs of every cloud provider. The tips and techniques are great regardless of the actual performance on a Micro (and applies to anyone trying to scale WordPress). Just wanted to clarify for anyone getting excited that they can now run their Fortune 500 website on a single AWS Micro that there are nasty surprises lurking just under the surface.
[1] http://docs.amazonwebservices.com/AWSEC2/latest/UserGuide/co...