There are several popular/hype theories and beliefs spread in the JEE world which we don't subscribe to in our web development: notably MVC and the tags/xml world in interface constructions. A further diffused superstition is the belief in code-based testing, or building application tests by writing further code (many share this skepticism, for example see webtest.canoo.com).

But this does not mean we do not believe in testing, on the contrary, testing is crucial in releasing reliable applications. And as Teamwork is getting wider adoption, not breaking adopted versions with new releases is becoming crucial for the survival of our product.

We have used extensively hallway testing; for us, a test makes sense if it is not done by the developers, and is performed on the user interface, not on the code. This is particularly true in the case of a web application, where the interface support changes in time and is not uniquely determined. Moreover the application behavior can be quite different depending on the browser, operating system and database used. To test an application like Teamwork, can mean testing several combinations for each page, hence resulting in several thousand tests, multiplied to the number of platform/db combinations. So we needed to somehow persist the tests done, and to make it possible to run thousands of them by one-click. We first checked

1. httpunit, but it’s based on the idea of writing tests in code, discarded,

and

2. htmlunit and watij.com , which rely on the assumption that you can automate finding links on a page, but this was partly true several years ago, though incredibly hard, and is impossible in a contemporary application like Teamwork which may any time alter the DOM by creating a link, so this too was discarded.

Going our way.

There are many many other tools, but the problem is how to ease the pain of recording structured information over Http request, and our framework knows how to do this for its own applications. So we extended our in-house practical-in-the-extreme development platform with a web based test recording and playing framework.

Our tests have a web based management interface, support composition, and are stored through Hibernate persistence on a separate database.

You can create and refer to variables, you can write assertions using the beanshell scripting language - we used bsh instead of JDK 6' inner scripting framework as the latter does not yes support Java-like syntax and is a bit unstable.
You may even get more reliable releases now :-D

Here are a few screenshots of our tool:

Labels: testing bugs