the goose

What could be more embarrassing than appearing on national TV, begging for investment?  I’m not sure.  Maybe having your website fall over at exactly when you have a chance to make some much needed sales would be the cherry on the cake.

I wondered whether I should do some performance testing on the site before the air date.  Piehole is written in rails, which has a name for poor performance.  That coupled with my slightly ropey programming expertise, could be a recipe for disaster.  On recommendation, I signed up for loadimpact.com and starting using their free account.

The page I tested was on this page.  This is what I got.

Holy crap. In English, that means I’m looking at a one minute loading time when there are 50 simultaneous people on the page.  I logged into my control panel and checked the memory usage and it seemed fine for the period of the test.  Something else was slowing things down other than a lack of memory.

My first step was to implement rails page caching which gave me a better performance.  This gave me about a 10% improvement but we were still looking at a very unacceptable page loading time.  Bear in mind, the average user will tolerate a 10 second wait – never mind one minute.

My next step was to see exactly where the bottleneck was.  Given that I was now serving up a plain HTML page but experiencing big delays, and there was no significant memory issue, it had to be the page contents that were holding things up.  I ran the same test on an empty test file and got this result.

Phew.  At least these numbers are a bit more acceptable.  The declining graph indicates that the server isn’t really being taxed at those load levels.  Lovely.  Now if I can just persuade the server to upload something more useful than an empty file.

I created a shorter version of our front page without much of the graphics.  Again, I got another acceptable declining response time.

I loaded up Y-Slow and started following its recommendations dutifully.  I spent some time fiddling with youtube videos (placing a loading image over them rather than loading the video on page load) but I was starting to hit a wall in terms of any improved performance  The load times were still way over what was acceptable for any page that had more than a couple of images on it.   I searched through the rest of the server and optimised some queries in other webapps on the same box.  In truth, I need not have bothered.

Rails does support moving your fixed assets (stylesheets, images, javascript resources) to a separate location so long as you use the image_tag to place images in webpages.  You can then specify an alternate server to serve your assets from.  I started off with using Amazon’s S3 service.

Bingo.  A 90% improvement.  The graph declines which means that we need to load it some more to find out what the upper limts are.  The only problem with using s3 is that it isn’t really designed as a content management network.  For one, it won’t compress css and javascript resources with gzip.  It also don’t add the correct MIME type headers and it is a bit of a pain to remember when to set the right permissions.

Next up I signed up for a $15 a month account with cachefly.com.  This bought me another 50% improvement to about a 1.5 second load time for 50 concurrent users.  So what did I learn?

1. Its easy to blame rails.  Give a dog a name and you’re likely to jump to conclusions – if that isn’t too much of a mixed metaphor.
2. Bundling javascript files, stylesheets, adding compression on your server to resources, yada yada, yada is all very well but if you spend more on wine than server costs each month – your easiest upgrade is probably to use a CDN.  It really isn’t as scary as it sounds.
3. Cachefly doesn’t seem to allow you to set the expir-header for resources.  This is a little annoying as it messes up your nice neat Y-Slow score but the performance is what matters – and it seems to be solid.
4. Loadimpact really is a great tool for identifying problems with loading times.

Of course this will all look rather redundant if the site falls over on Thursday night.