Selenium Selenese Continuous Integration Runner

on GitHub in the hopes that it will save other people time when trying to get their Selenese tests running from a continuous integration server. It’s very simple but one thing I battled with was that I had to patch the selenium JAR to get it to work with Firefox 3.0. It should work fine in any continuous integration server regardless if it’s Hudson, Cruise, Cruise Control, Bamboo, etc.

The functional testing products I’ve used that drive a real browser include Test Complete (commercial), Selenium, and Watir. I think all 3 do a good job but one thing I like about Selenium is that it’s dirt simple to get a user productive with the Selenium IDE Firefox plugin. However, that benefit is also the most limiting factor of the Selenium IDE which is that to be able to re-open tests in Selenium IDE you have to save them as Selenese (which is the most limited of the testing languages that Selenium supports). Still, I think Selenese is a reasonable choice for a lot of organizations that need a moderately sophisticated functional test suite.

1. Rails is a one stop shop with a web MVC framework that has AJAX support, an OR Mapping framework, logging framework, default directory structure, unit testing, functional testing, etc… Bottom line is that it’s much easier for someone to get productive quickly in RoR learning good practices such as TDD than in any equivalent Java setup. Heck, at work we still don’t have a good functional web testing framework for the programmers to use because after some tinkering I still couldn’t get it to start Tomcat, deploy, and run the HttpUnit tests reliably. The first time I tried Rails, the logging, mvc framework, OR Mapping, web services integration, functional tests, unit tests, etc… just worked, wow!
2. Rails let’s you learn things when you’re ready to whereas in Java, just to get my projects off the ground, I have to understand the build system, Hibernate configuration files, web.xml, log4j configuration, my MVC configuration, etc… I still don’t know the first thing about Rake (the build system) even though I’ve built a couple small web projects in RoR and you know what, that’s a real testament to how well it does it’s job. I’m focusing on solving my business problem now, not on learning a build system! Similarly there are massive black holes in my knowledge around ActiveController, ActiveRecord, Rails’ AJAX support, etc… but it hasn’t stopped me from using those features and being productive! I hope to learn the ins and outs of these components eventually but that’s in direct contrast to Java where you have to front-load tons of knowledge to be productive!
3. Rails recognizes that in the real world there are differences between your dev, test, and production environment whereas I’ve always had to roll my own solution in Java whether it’s through JNDI or the creative use of property files. The key here is that in Rails you have all of those settings in your project under source control making it easy to lookup how a setting differs between test and production, with JNDI that information is stored on the server but it’s irrelevant because everyone does it differently anyhow.
4. The Rails community recognized the need for a production deployment solution and came out with Capistrana (ex SwitchTower) whereas in Java with Tomcat we had to write our own from scratch that goes to our 10 application servers, takes each one out of production, deploys the WAR, performs a test, and then brings it back into production before moving onto the next server.
5. Jumping in on an existing Rails project should be easy because there are strong conventions whereas in Java every project does almost everything a little differently. You have to learn the Ant targets, the project source layout, how dependencies are managed, how they deploy into Tomcat, and so on. Maven certainly helps address this issue in the Java space but it’s adoption rate is low compared to Ant.

That said, there are still things I really like about Java. For starters I’m an Emacs to IDE convert and nothing compares to the refactoring and code analysis capabilities of JetBrains Idea. Since Ruby is dynamically typed, no IDE will ever support the degree of refactoring we enjoy in Java. Also, once you have your Java environment setup and everything integrated and working, developing new web pages is fast. For example when I develop in Idea I can change a class or a web page and hit refresh in my browser just like I do in Ruby. Since we use JSPX and TAGX files, IDEA tells me if my XHTML is not well-formed XML… I could go on and on.

Suffice it to say that I still like Java but I’m loving that there’s some fierce competition on the block and I hope someday to have the opportunity to develop a large software project in Rails. If you still have your doubts about Ruby on Rails, read Agile Web Development with Rails, it’s a great book that will knock your socks off!

Related reading: Keith Pitty looks at Java versus Ruby on Rails

My only complaint with the book is one sentence where they recommend feature boxed over time boxed releases which I generally disagree with.

Part of that process involves doing performance reviews to check-in a few times a year, get feedback from their colleagues, give raises, evaluate progress, identify areas for improvement or professional development, etc… On the whole I think the review process is a good thing to formally touch base but it should be pretty light weight and not too time consuming. After all, I’d rather have my team members working on adding value to our customers than spending days writing self evaluations.

The discussion inevitably arises around SMART (Specific, Measurable, Achievable, Realistic, and Time bound) objectives. In some cases they are obvious and make a lot of sense to me such as X% uptime for a sysadmin or Y% unit test coverage for the programmers. However, when you start trying to make all objectives SMART it can work against the organization! Like measuring lines of code, defects filed in the bug tracking system, achieving Z hours of logged work per developer per release, etc… These are all numbers that encourage employees in the wrong direction, such as asking people not to file bugs in the bug tracking system, inflating actual hours worked, etc…

This comes to my final point. Trust! Many of the most meaningful objectives for an employee or organization are not SMART but that’s OK. An organization needs to trust it’s managers to be able to evaluate their people on non measurable (subjective) objectives. After all that keeps an organization or employee more agile to respond to change. Quite frankly if a manager can’t review an employee simply based on gut instinct and peer review, then there’s a problem! Don’t get me wrong, I understand the value of SMART objectives, they just need to be applied in moderation in the areas where they are appropriate!

Joel has also blogged on why you should scrap performance reviews.

1. Feature boxed releases allow you to focus on getting the features the business side needs in the release and getting them right. However feature boxing is more easily prone to scope creep, continually pushing the release date, and spending too much time getting a feature “just” right instead of getting it in front of customers.

2. Time boxed iterations continuously keep the organization focusing on what’s most important and reliably get things done on a certain date at the cost of cutting features when estimates turn out to have been to low. You also spend a little less time actually implementing and more time planning, estimating, and meeting.

I think a preference for one or the other is largely a philosophical question. Do you value releasing early and often or correctness and completeness?

Having worked in both environments I strongly favor time boxed iterations because they provide a very light weight yet key form of discipline that many organizations lack including:

1. Getting the organization to make tough prioritization decisions early
2. Nipping scope creep in the bud because when projects start to run over developers start to make noise when a deadline is looming
3. It gets people thinking about breaking features down into per release iterations that they can get in front of customers quickly

The toughest part for many organizations is that time boxing requires a management team that’s willing to make difficult feature cutting/prioritization decisions instead of asking employees to work overtime. I think that’s why time boxing is often looked upon negatively by developers because they associate it with overtime.

At GreatSchools we do 2 week time boxed releases (although on occasion features will span a release or two) and we use Jira extensively for tracking each feature/project including estimates against actuals. So far the system has been working relatively well and has been extremely beneficial as a business planning and prioritization tool. Releasing often has also had the added benefit of forcing us to improve our processes and tools in the areas of release engineering, continuous integration, unit and functional testing, etc…

How do you manage releases at your organization and how has it been working for you?