Here’s how we went about using a private S3 bucket to store our Git repository of Chef Recipes. Thanks to this post on using JGit to publish to S3 which got us started, the key difference is we wanted to use a private S3 bucket and it took us some experimenting to figure out how to update an existing Git repo (via fetch and merge) from S3.

Download jgit.sh, rename it to jgit and put it in your path (for example $HOME/bin).

Setup the .jgit config file and add the following (substituting your AWS keys):

vim ~/.jgit

Note, by not specifying acl: public in the .jgit file, the git files on S3 will be private (which is what we wanted). Next create an S3 bucket to store your repository in, let’s call it git-repos, and then create a git repository to upload:

In the above I’m using the s3cmd command line tool to create the bucket but you can do it via the Amazon web interface as well. Now let’s push it up to S3 (notice how we use jgit whenever we interact with S3, and standard git otherwise):

Now go somewhere else (e.g. cd /tmp) and try cloning it:

When it comes time to update it (because jgit doesn’t support merge or pull) you do it in 2 steps:

Sometimes you inherit a non Ruby based web app written in PHP, Perl, Java, C#, or Python and you want to create an automated functional/acceptance test suite to minimize the amount of manual QA you need to do. Selenium is a great tool for doing just that and at Common Sense Media I created the Selenium Continuous Integration Runner to make it easy to run Selenese based tests from your Continuous Integration Server such as Hudson, Cruise Control, or Teamcity. That said, Selenese based tests leave something to be desired in terms of readability and maintainability.

The Ruby camp is often at the forefront of great testing frameworks with Aslak Hellesøy often leading the charge with innovations like RSpec and Cucumber. I’d read and played around with Cucumber on Rails based apps and was loving the concept of english language specs that are actually executable as well. But when my friend and former colleague Chris Kimm mentioned that the testing group at his company was using it to test their Java application, I started to get excited as that was the problem I was about to try and tackle at VolunteerMatch.

However, getting the project structure in place required a little help from the Cucumber mailing list and in the hopes of saving others the time, I’ve published the Standalone Cucumber Test Suite to Github. It’s simply a Cucumber project designed to test a non Ruby web application with a sample search.feature that uses Google search as the basis for the test. It uses Capybara with the Selenium Webdriver on the backend but you can also switch it to use Culerity or Webrat if you don’t want it actually firing up a real browser to run the tests. Fork it or download it to start testing your existing website with Cucumber!

One of the jobs of a good programmer is to make their code readable and in this case by naming the variable sum it’s obvious what’s going on to the right so if you can simplify the anonymous function or use syntactic sugar to reduce the amount of code there’s a good chance you can improve the readability as follows:

Then even if the programmer reading the code is coming from another language and doesn’t know :+ or _+_ you’ll still have improved readability because they certainly understand how a sum is computed and can move past the line of code instead of having to read 3-4 lines of a for loop for something as simple as a summing operation. I love the balance of being very concise with the simple stuff so that I can be more verbose with the complex stuff, whereas in Java even the simple stuff is often extremely verbose.

There is of course the extreme of making your code too concise. Look no further than write-only languages like Perl to see where too much conciseness and obfuscation can lead to unmaintainable code.

This example is only trivial and somewhat contrived but I find that newer languages like Ruby and Scala give you significantly better tools to elegantly dial up or down the conciseness than Java or C# resulting in less code + more readable code = easily maintainable code.

As an example, I used just about every conciseness trick in the book I knew of when composing my submission for the 3rd Ruby Challenge’s Shortest Path Algorithm and I only discarded the ones that I thought obfuscated the intent. I like to think the reason my submission won was precisely because it was concise while still being readable. Writing that same algorithm with the same level of readability in Scala would be easy but in Java it would have tripled the lines of code hurting readability somewhat.

Selenium Selenese Continuous Integration Runner

on GitHub in the hopes that it will save other people time when trying to get their Selenese tests running from a continuous integration server. It’s very simple but one thing I battled with was that I had to patch the selenium JAR to get it to work with Firefox 3.0. It should work fine in any continuous integration server regardless if it’s Hudson, Cruise, Cruise Control, Bamboo, etc.

The functional testing products I’ve used that drive a real browser include Test Complete (commercial), Selenium, and Watir. I think all 3 do a good job but one thing I like about Selenium is that it’s dirt simple to get a user productive with the Selenium IDE Firefox plugin. However, that benefit is also the most limiting factor of the Selenium IDE which is that to be able to re-open tests in Selenium IDE you have to save them as Selenese (which is the most limited of the testing languages that Selenium supports). Still, I think Selenese is a reasonable choice for a lot of organizations that need a moderately sophisticated functional test suite.

Update 7/8/2009: With the latest batch of contributed patches standalone migrations now works just like Rails migrations
Update 12/26/2008: I switched standalone migrations to use a Rakefile instead of a Ruby script.

In my work managing websites I end up working in Ruby, Java, and PHP. In everything but Rails managing the schema requires rolling your own solution. As a result I’ve started using Rails migrations in non-Rails projects to manage the schema. It’s not much code but I figured others might benefit from it so I created a little Github project called standalone migrations.

It’s based on Lincoln Stoll’s blog post titled Stand-alone ActiveRecord migrations and David Welton’s blog post titled Using Migrations Outside of Rails.

Assuming you have Ruby and Gem installed on your machine, here’s how to use it:

gem install -y activerecord rake mysql
wget http://github.com/thuss/standalone-migrations/zipball/master (or fetch it using git)
unzip it, and mv to something like my_non_rails_project/db
cd my_non_rails_project/db/ (or wherever you put it)
cp config/database_sample.yml config/database.yml
vi config/database.yml
./new_migration some_user_story
vi migrations/*_some_user_story.rb
rake db:migrate (this applies your newly created migration)

I’m pleased to say that after 3 hours I had the install done and all of my software installed and working including Jetbrains IDEA, Netbeans, Skype, VirtualBox, Flash plugin, Java plugin, Web Developer Toolbar, and Firebug. Apparently things have gotten a little better for 64 bit Linux over the past year and a half! However, I’m still running into odd issues every now and again related to both Sun’s 64 bit Java (IDE and Jetty occasionally hangs) and the the Open JDK browser plugin which is still buggy. Hopefully future updates will help address these issues. Anyhow, I thought I’d throw out the links I used to get everything working:

Fonts
Howto: Install Mac Fonts on Ubuntu
Make Firefox in Ubuntu look much better

Skype
Add the Medibuntu repository
sudo apt-get install skype-static

Flash plugin
Install Flash 10 Under Ubuntu Linux 64 bit Edition

Java plugin
sudo apt-get install openjdk-6-jre icedtea-gcjwebplugin

It’s exciting to see the convergence of my two favorite languages coming along!

require 'irb/completion'
ARGV.concat [ "--readline", "--prompt-mode", "simple" ]
IRB.conf[:EVAL_HISTORY] = 1000
IRB.conf[:SAVE_HISTORY] = 1000
IRB.conf[:HISTORY_FILE] = File::expand_path("~/.irbhistory")

Now when I fire up irb or script/console I get full readline support including tab completion (for example type Time:: and hit tab), reverse command search (control-R), and saved command history so when I exit and restart my command history is still there!

List states = new ArrayList();
state.add(State.CA);
state.add(State.WY);

Then today I read about using an anonymous subclass with instance initializer to initialize a collection which appeals to the block loving Rubyist in me:

List states = new ArrayList() {{ add(State.CA); add(State.WY); }};

Another approach shown to me by a new co-worker specific to Lists (that involves creating an extra array) but is also quite short is:

List states = Arrays.asList(new State[]{State.CA, State.WY});

Update: Ricky Clarkson pointed out the best solution so far in my opinion (requires JDK 5 or higher) using varargs and generics:

List<String> list=Arrays.asList("rocks","my","world");

I had one minor problem and it was that we connect to multiple databases so we have multiple session factories that interact with EhCache. Hibernates org.hibernate.cache.EhCacheProvider now expects one configuration per sesssion factory which I wasn’t too excited about given we have 51 databases (1 database per state plus Washington DC) and it’s been working fine for 2 years with one cache provider. The problem caused this exception:

 Invocation of init method failed; nested exception is java.lang.NullPointerException
    [java] Caused by: org.springframework.beans.factory.BeanCreationException:
       Error creating bean with name 'surveysSessionFactory' defined in class path resource
          [gs/data/dao/hibernate/applicationContext-hibernate.xml]:
          Invocation of init method failed;
          nested exception is java.lang.NullPointerException
    [java] Caused by: java.lang.NullPointerException
    [java] at net.sf.ehcache.CacheManager.detectAndFixDiskStorePathConflict(CacheManager.java:269)
    ...

However, I wasn’t the only person who had this problem so a quick search turned up that EhCache now offers the net.sf.ehcache.hibernate.SingletonEhCacheProvider and switching the hibernate.cache.provider_class property to this new provider solved the problem and allows us to continue using a single cache provider.