As some users have discovered the hard way, if your Gmail account gets compromised and your email is deleted, you’ll likely never get it back unless you have an off-site (outside of Google) backup.

One great service that does this for you is Backupify and their free plan will backup mailboxes up to 1GB. I personally love cloud data but prefer local backups so I use Getmail which is a little Python script to download my email. Here’s a more detailed post on how to setup Getmail by Matt Cutts. That said, I have my script backup my entire mailbox (instead of just my inbox) and I use IMAP instead of POP so here’s my getmail.gmail config file:

Here’s how we went about using a private S3 bucket to store our Git repository of Chef Recipes. Thanks to this post on using JGit to publish to S3 which got us started, the key difference is we wanted to use a private S3 bucket and it took us some experimenting to figure out how to update an existing Git repo (via fetch and merge) from S3.

Download jgit.sh, rename it to jgit and put it in your path (for example $HOME/bin).

Setup the .jgit config file and add the following (substituting your AWS keys):

vim ~/.jgit

Note, by not specifying acl: public in the .jgit file, the git files on S3 will be private (which is what we wanted). Next create an S3 bucket to store your repository in, let’s call it git-repos, and then create a git repository to upload:

In the above I’m using the s3cmd command line tool to create the bucket but you can do it via the Amazon web interface as well. Now let’s push it up to S3 (notice how we use jgit whenever we interact with S3, and standard git otherwise):

Now go somewhere else (e.g. cd /tmp) and try cloning it:

When it comes time to update it (because jgit doesn’t support merge or pull) you do it in 2 steps:

To that end one of the tools I want to give a shout out to is the MySQL Performance Tuning Primer Script. You download and run it against a production system (that has preferably been running under normal load for a day or two so that it’s gathered stats). It’s a read-only script so you don’t need to worry about it changing anything but it makes some great recommendations about which tuning parameters may need adjustment. Here’s a snippet from from a production server:

KEY BUFFER
Current MyISAM index space = 173 M
Current key_buffer_size = 1 G
Key cache miss rate is 1 : 1003
Key buffer fill ratio = 7.00 %
Your key_buffer_size seems to be too high.
Perhaps you can use these resources elsewhere
…
…
…

It’s no replacement for a DBA but if you want to get a somewhat sane my.cnf going for your particular application it’s a great place to start!

Update 7/8/2009: With the latest batch of contributed patches standalone migrations now works just like Rails migrations
Update 12/26/2008: I switched standalone migrations to use a Rakefile instead of a Ruby script.

In my work managing websites I end up working in Ruby, Java, and PHP. In everything but Rails managing the schema requires rolling your own solution. As a result I’ve started using Rails migrations in non-Rails projects to manage the schema. It’s not much code but I figured others might benefit from it so I created a little Github project called standalone migrations.

It’s based on Lincoln Stoll’s blog post titled Stand-alone ActiveRecord migrations and David Welton’s blog post titled Using Migrations Outside of Rails.

Assuming you have Ruby and Gem installed on your machine, here’s how to use it:

gem install -y activerecord rake mysql
wget http://github.com/thuss/standalone-migrations/zipball/master (or fetch it using git)
unzip it, and mv to something like my_non_rails_project/db
cd my_non_rails_project/db/ (or wherever you put it)
cp config/database_sample.yml config/database.yml
vi config/database.yml
./new_migration some_user_story
vi migrations/*_some_user_story.rb
rake db:migrate (this applies your newly created migration)

I’m pleased to say that after 3 hours I had the install done and all of my software installed and working including Jetbrains IDEA, Netbeans, Skype, VirtualBox, Flash plugin, Java plugin, Web Developer Toolbar, and Firebug. Apparently things have gotten a little better for 64 bit Linux over the past year and a half! However, I’m still running into odd issues every now and again related to both Sun’s 64 bit Java (IDE and Jetty occasionally hangs) and the the Open JDK browser plugin which is still buggy. Hopefully future updates will help address these issues. Anyhow, I thought I’d throw out the links I used to get everything working:

Fonts
Howto: Install Mac Fonts on Ubuntu
Make Firefox in Ubuntu look much better

Skype
Add the Medibuntu repository
sudo apt-get install skype-static

Flash plugin
Install Flash 10 Under Ubuntu Linux 64 bit Edition

Java plugin
sudo apt-get install openjdk-6-jre icedtea-gcjwebplugin

My first attempt to keep my Ruby on Rails sites running at Dreamhost involved modifying dispatch.fcgi to implement the frao_handler approach as described on the wiki. This left me with a 3% 500 error rate. Ughh!

My second attempt involved gracefully restarting dispatch.fcgi’s every hour or two to avoid the wrath of the Dreamhost process monitor. This reduced the 500 error rate down to 0.2% which was still too high!

I’ve finally gotten my 500 error rate down to 0 by making some minor changes to the signal handling code of dispatch.fcgi. If the dispatch.fcgi process is in the midst of handling a request I defer letting it be killed until the request is complete. I did this by installing a custom TERM signal handler that protects the dispatch.fcgi process while a request is being processed. This solution works for my Rails 1.1 and Rails 1.2 (previously Edge) based sites. Here’s my complete dispatch.fcgi:

#!/usr/bin/ruby1.8

require File.dirname(__FILE__) + "/../config/environment"
require 'fcgi_handler'

class RailsFCGIHandler
 private
   def busy_exit_handler(signal)
     dispatcher_log :info, "busy: asked to terminate during request signal #{signal}, deferring!"
     @when_ready = :exit
   end

   # Dreamhost sends the term signal and if we're handling a request defer it
   def term_process_request(cgi)
     install_signal_handler('TERM',method(:busy_exit_handler).to_proc)
     Dispatcher.dispatch(cgi)
   rescue Exception => e  # errors from CGI dispatch
     raise if SignalException === e
     dispatcher_error(e)
   ensure
     install_signal_handler('TERM', method(:exit_now_handler).to_proc)
   end
   alias_method :process_request, :term_process_request
end

RailsFCGIHandler.process!

This makes it totally inadequate for running a production database server. On the forums people have argued for using replication or clustering solutions while others have advocated streaming redo logs to Amazon’s S3 service. However, let’s look at the disaster recovery scenario if all your virtual machines go down (and don’t think it can’t happen, racks overload their circuits, generators fail during a power outages, AC’s can’t keep up during heat-waves, etc…):

- With a VPS or dedicated server you startup your servers, do a data integrity check, and you’re up and running again!
- With EC2 you restart the database server image, then restore the full database backup from the previous night (which you’ve been backing up to S3), then you rerun all of the redo logs (which you’ve also been backing up to S3).

The latter scenario isn’t that bad with a small database but for with a large database it could add hours to your recovery time. I imagine Amazon will eventually add permanent storage to EC2 and if/when that happens you’d be able to compare Amazon EC2 to a VPS but right now it’s comparing apples and oranges!

In my personal experience partitioning a disk into root, boot, var, home, usr/local, tmp, var/mysql, etc… is generally overkill and has caused me far more headaches than it has prevented problems. Eventually you end up having one filesystem with gigabytes free that you really need for another filesystem which leads you to do a dump-repartition-restore. And that’s always a hassle if the machine is in use by real users during business hours so you end up having to work a night or weekend.

So, while I still like to have 4 partitions on heavy duty production servers (root, var, app-or-user-data, swap), I no longer bother with desktop, development, or light use production servers. I prefer the ultra simple 2 partition scheme of root and swap and since I started doing that 3 years ago I’ve had to do far fewer dump-repartition-restores.