Todd Huss

Hello lazy web, I’m looking for some advice on caching proxy servers and thought you might have some good pointers!

We currently front our site with 3 caching proxy servers to offload static content from the web servers and we haven’t found the right solution yet. We’ve tried a number of solutions including Oops and Apache with mod_cache (which we’re currently running) but they are either missing a critical feature, have buggy caching behavior, or crash under heavy load. The main features I’m looking for are:
(more…)

Update 2/11/07: Just to clarify, this plugin won’t keep your Rails site running on Dreamhost if the sum of all your running processes exceeds 200MB (which is when their process monitor kills your processes). Run ps aux and sum the VSZ column and then divide by 1024. If you’re pushing the limit and are only running one Rails site you may want to look at another provider. If you’re running multiple Rails sites, run each site under a different user (one account can create many users).

Per Thomas Aylott’s suggestion on my previous post Keeping Rails Running at Dreamhost Part 2 I’ve decided to release my Dreamhost signal handling code as a Rails plugin. I’m using it on a Rails 1.2.1 site and a Rails 1.1.6 site.
(more…)

Update 2/2/07: Per Thomas’ comment I’ve released the code below as the dreamhost rails plugin.
Update 1/25/07: People have reported difficulties copy and pasting the dispatch.fcgi source code from this blog post so here is a dispatch.fcgi to download. Make sure you make it executable!

My first attempt to keep my Ruby on Rails sites running at Dreamhost involved modifying dispatch.fcgi to implement the frao_handler approach as described on the wiki. This left me with a 3% 500 error rate. Ughh!
(more…)

Update 1/12/07: This solution has reduced my 500 error rate down to less than 0.2% but it’s still not perfect. I continue to plug away at this…

Update 1/24/07: I’ve finally solved my Dreamhost 500 error problems with a different solution described in Keeping Rails running at Dreamhost part 2

This weekend I moved two of my newer Ruby on Rails based sites (http://windandtides.com and http://gearandboats.com) over to Dreamhost. You simply can’t beat their prices for hosting a small site (if you prepay for 2 years and use coupon code GABRITO to save $50 the total is $140.80) and I’ve found their customer service to be pretty responsive too.
(more…)

I’ve been reading all the great things about Amazon EC2 (or Elastic Compute Cloud) and lots of pricing comparisons with VPS and dedicated hosting. I finally got an EC2 account and tinkered a bit and there’s a big difference between EC2 and Virtual Private Server or Dedicated hosting that most of the preliminary write-ups I’ve seen completely overlook.
(more…)

Most systems administrators will tell you it’s important to partition your install into anywhere from 4-7 discrete patitions (or slices if you’re in the BSD camp). While I think it’s good advice in certain cases, the headaches of mis-guessing disk space requirements have bitten me so many times that I’ve grown jaded and only create a new partition for a filesystem if there’s a really good reason such as:
(more…)

We’re hiring a senior MySQL and FreeBSD systems administrator to work at GreatSchools (a 40 person company) in San Francisco. If you or someone you know is interested in working on a high traffic website (1M page views per day) consisting of 18 FreeBSD servers, redundant load balancers, MySQL 5 with replication and so on send your resume to jobs-sysadmin at greatschools dot net. If you have a family or want quality of life and are interested in working a compressed week (4 days) or a reduced week (e.g. 32 hours) that’s a possibility too given the right candidate!

At GreatSchools we do around 1M real page views per day and another 250k or so for crawlers. Before content compression we were running well in excess of 10Mbit/s during peak hours and were getting hit with bursting charges on high traffic months. When we switched our proxy servers to Apache with mod_deflate (gzip based compression) we saw a 35% decrease in bandwidth utilization and the 3 proxy servers that do the compression and sit in front of our 10 web servers barely register a load at all.
(more…)

I’ve been using Subversion for just about everything I work on including keeping my documents synchronized across multiple computers, backups, source control, etc… I got the idea a while back from Martin Fowler’s Bliki who uses it for a similar purpose. All you need to host your own Subversion repository is a shell account on a machine that you can SSH into. Sure, you could create a repository on your local machine (more…)

As I’ve described before, MySQL has some appalling out of the box settings which will thwart your attempts at good data integrity! They’ve clearly seen the light though and at least give you an option to achieve good data integrity through foreign keys in InnoDB and proper handling of non-nullable fields and invalid values through the servers sql-mode setting.

To make matters worse it seems MySQL may have different out of the box settings per platform and per version. For example a friends 5.0.21 install on Windows XP had the sql-mode set with relatively strict settings whereas 5.0.18 on my Mac had an empty sql-mode. This only illustrates the necessity of always explicitly defining these settings with each install yourself so that you get consistent behavior.
(more…)