Todd Huss » FreeBSD

Production MySQL performance tuning

Todd Huss — Fri, 16 Jan 2009 04:22:00 +0000

For the past 9 years I’ve been working almost exclusively with MySQL (with a little PostgreSQL thrown in) and while I don’t do nearly as much DBA work these days, I still find myself troubleshooting a query or tuning my.cnf. While in general I find MySQL to be a lot more straightforward to work with, it’s still equally important to tune it for your applications needs.

To that end one of the tools I want to give a shout out to is the MySQL Performance Tuning Primer Script. You download and run it against a production system (that has preferably been running under normal load for a day or two so that it’s gathered stats). It’s a read-only script so you don’t need to worry about it changing anything but it makes some great recommendations about which tuning parameters may need adjustment. Here’s a snippet from from a production server:

KEY BUFFER
Current MyISAM index space = 173 M
Current key_buffer_size = 1 G
Key cache miss rate is 1 : 1003
Key buffer fill ratio = 7.00 %
Your key_buffer_size seems to be too high.
Perhaps you can use these resources elsewhere
…
…
…

It’s no replacement for a DBA but if you want to get a somewhat sane my.cnf going for your particular application it’s a great place to start!

Comparing Amazon EC2 with VPS and dedicated hosting

Todd Huss — Fri, 01 Dec 2006 05:01:21 +0000

I’ve been reading all the great things about Amazon EC2 (or Elastic Compute Cloud) and lots of pricing comparisons with VPS and dedicated hosting. I finally got an EC2 account and tinkered a bit and there’s a big difference between EC2 and Virtual Private Server or Dedicated hosting that most of the preliminary write-ups I’ve seen completely overlook.

- With a VPS or Dedicated server you can shutdown, reboot, crash, or have an outage and your data stays on disk.
- With an Amazon EC2 account when you shutdown, reboot, crash, or have an outage any new data that was not in the server image that you originally uploaded is gone!

This makes it totally inadequate for running a production database server. On the forums people have argued for using replication or clustering solutions while others have advocated streaming redo logs to Amazon’s S3 service. However, let’s look at the disaster recovery scenario if all your virtual machines go down (and don’t think it can’t happen, racks overload their circuits, generators fail during a power outages, AC’s can’t keep up during heat-waves, etc…):

- With a VPS or dedicated server you startup your servers, do a data integrity check, and you’re up and running again!
- With EC2 you restart the database server image, then restore the full database backup from the previous night (which you’ve been backing up to S3), then you rerun all of the redo logs (which you’ve also been backing up to S3).

The latter scenario isn’t that bad with a small database but for with a large database it could add hours to your recovery time. I imagine Amazon will eventually add permanent storage to EC2 and if/when that happens you’d be able to compare Amazon EC2 to a VPS but right now it’s comparing apples and oranges!

Trafeoffs of aggressive filesystem partitioning

Todd Huss — Thu, 24 Aug 2006 00:27:44 +0000

Most systems administrators will tell you it’s important to partition your install into anywhere from 4-7 discrete patitions (or slices if you’re in the BSD camp). While I think it’s good advice in certain cases, the headaches of mis-guessing disk space requirements have bitten me so many times that I’ve grown jaded and only create a new partition for a filesystem if there’s a really good reason such as:

1. Performance
2. Preventing an application, user, or logging subsystem from filling up the whole disk
3. To keep the operating system and data separate (which eases upgrades, restores, etc…)
4. If I have multiple disks or RAID arrays and can’t or don’t want to use an LVM

In my personal experience partitioning a disk into root, boot, var, home, usr/local, tmp, var/mysql, etc… is generally overkill and has caused me far more headaches than it has prevented problems. Eventually you end up having one filesystem with gigabytes free that you really need for another filesystem which leads you to do a dump-repartition-restore. And that’s always a hassle if the machine is in use by real users during business hours so you end up having to work a night or weekend.

So, while I still like to have 4 partitions on heavy duty production servers (root, var, app-or-user-data, swap), I no longer bother with desktop, development, or light use production servers. I prefer the ultra simple 2 partition scheme of root and swap and since I started doing that 3 years ago I’ve had to do far fewer dump-repartition-restores.