Keeping Ruby on Rails running at Dreamhost

Update 1/12/07: This solution has reduced my 500 error rate down to less than 0.2% but it’s still not perfect. I continue to plug away at this…

Update 1/24/07: I’ve finally solved my Dreamhost 500 error problems with a different solution described in Keeping Rails running at Dreamhost part 2

This weekend I moved two of my newer Ruby on Rails based sites (http://windandtides.com and http://gearandboats.com) over to Dreamhost. You simply can’t beat their prices for hosting a small site (if you prepay for 2 years and use coupon code GABRITO to save $50 the total is $140.80) and I’ve found their customer service to be pretty responsive too.

I followed their Wiki based Ruby on Rails instructions and had the sites up and running with Capistrano in no time. However, I soon discovered that their process monitor likes to kill off long running processes (effectively taking my sites offline without warning) and the only indication of what’s wrong is this error in the apache error.log:

FastCGI: comm with (dynamic) server "/path/to/dispatch.fcgi" aborted: (first read) 
  idle timeout (60 sec), referer: http://gearandboats.com
FastCGI: incomplete headers (0 bytes) received from server "/path/to/dispatch.fcgi", 
  referer: http://gearandboats.com

To fix this I wrote a short ruby script fcgimaint.rb that I run once per hour via cron to restart my dispatch.fcgi processes (first with a USR1 and then a kill -9 if the USR1 didn’t work). This has the added benefit of cleaning up accumulated FastCGI processes and starting with a fresh memory footprint in case of a memory leak:

Crontab entry: 33 * * * * /home/myuser/bin/fcgimaint.rb

#!/usr/bin/ruby
def dispatchers()
  ps = `ps ux`.grep(/dispatch\.fcgi/) 
  ps.map {|p| {'pid' => p.split[1], 'start' => p.split[8]}}
end
dispatchers.each { |d| `kill -USR1 #{d['pid']}` }
sleep 30
# Kill dispatchers that didn't restart nicely
dispatchers.each do |d| 
  if (Time.now.hour != d['start'][0..1])
    `kill -KILL #{d['pid']}`
  end
end

One other tool I found useful (especially during this process when the sites were going down periodically) was using Geoffrey Grosenbach’s modifications to dwatch to monitor my sites and email my cell phone if there was a problem.

Caveat: if your site does more than 10,000 page views per day then a shared hosting plan is not the answer, you should be looking at dedicated or VPS hosting.