Sunday, January 07, 2007
Testing new Teamwork releases
But this does not mean we do not believe in testing, on the contrary, testing is crucial in releasing reliable applications. And as Teamwork is getting wider adoption, not breaking adopted versions with new releases is becoming crucial for the survival of our product.
We have used extensively hallway testing; for us, a test makes sense if it is not done by the developers, and is performed on the user interface, not on the code. This is particularly true in the case of a web application, where the interface support changes in time and is not uniquely determined. Moreover the application behavior can be quite different depending on the browser, operating system and database used. To test an application like Teamwork, can mean testing several combinations for each page, hence resulting in several thousand tests, multiplied to the number of platform/db combinations. So we needed to somehow persist the tests done, and to make it possible to run thousands of them by one-click. We first checked
1. httpunit, but it’s based on the idea of writing tests in code, discarded,
2. htmlunit and watij.com , which rely on the assumption that you can automate finding links on a page, but this was partly true several years ago, though incredibly hard, and is impossible in a contemporary application like Teamwork which may any time alter the DOM by creating a link, so this too was discarded.
Going our way.
Our tests have a web based management interface, support composition, and are stored through Hibernate persistence on a separate database.You can create and refer to variables, you can write assertions using the beanshell scripting language - we used bsh instead of JDK 6' inner scripting framework as the latter does not yes support Java-like syntax and is a bit unstable.
You may even get more reliable releases now :-D
Here are a few screenshots of our tool:
Labels: testing bugs