Pramatr Blog

Whilst performing code reviews recently, one of the major tasks was reviewing the tests accompanying the code. One of the most surprising discoveries, was the lack of attention paid to the tests and how regularly this was becoming an issue throughout the test code.

Test code is just as likely to contain bugs as the code it is supposed to be testing. After all, if the tests are just code, why wouldn’t it contain bugs. The possibilty of it containing bugs could actually be higher as it doesn’t have tests and in many teams isn’t treated with the same respect as the rest of the code. Tests must therefore be kept as simple and easy to understand as possible and employ the same tools and techniques available when writing tests as are used in the rest of the code.

PMD and FindBugs are great tools to identify potential problems within code, but I have only ever seen one project that applies them to the test code. PMD/CPD is a great tool to identify duplicate code blocks, but I have never seen a project apply it to test code. If duplicate code isn’t acceptable, why should it be tolerated in test code?

@Test
public void updateFirstName() {
    ...
    User updated = getUserById(expected.getId());
    assertNotNull(updated);
    assertEquals(expected.getFirstName(), updated.getFirstName());
    assertEquals(expected.getMiddleName(), updated.getMiddleName());
    assertEquals(expected.getLastName(), updated.getLastName());
}

@Test
public void updateMiddleName() {
    ...
    User updated = getUserById(expected.getId());
    assertNotNull(updated);
    assertEquals(expected.getFirstName(), updated.getFirstName());
    assertEquals(expected.getMiddleName(), updated.getMiddleName());
    assertEquals(expected.getLastName(), updated.getLastName());
}

Over time, tests will inevitably suffer from the same problems as the rest of the code. Tests become bloated, complex, duplicated and hard to understand if they are not maintained and refactored regularly just like the rest of the code. The potential problem with refactoring tests is that you are refactoring something which does not have tests to verify your changes, but testing tests is a controversial issue and can probably be better solved by conducting thorough code reviews.

Code reviews seem to be an underrated technique within test code. Having spoken to many other developers, the majority of them had never participated in any test code reviews, with only a small number having used this approach on several occasions. What makes this even stranger, is the fact that many of these developers do conduct code reviews on a semi-regular basis. This is somewhat anecdotal evidence, but it would be very interesting to see some real numbers on this subject. How many teams conduct thorough code reviews of tests and treat it with the same importance as the rest of the code?

Code without documentation is considered by many to not be code complete, but tests without documentation are a pretty common occurrence. Tests should be simple, which means documentation is not necessarily required, but does that same argument apply to the rest of the code? Tests can have intent revealing names which are sufficient in describing the intended purpose of the test and projects like TestDox can take this and turn it into readable documentation. Is this documentation sufficient, if the test fails, does it provide enough of a clue to what was actually being tested?

In the past I have found test documentation to be extremely useful when writting integration tests. This documentation described the tests not in developer terms, but instead in a use case form. By writing a simple parser, it was possible to generate documentation from the tests which would allow the QA team to understand which flows had been executed. More importantly, it also allow them to quickly understand what a individual test was doing when it failed. This documentation was written as an experiment, but it has proved extremely useful to both developers and QA when something goes wrong.

When writing tests, teams need to ensure they don’t neglect this code. Tests should be maintained, refactored, reviewed and measured to ensure the same quality as they would in the rest of their code. Tests are still code.

Over the past few years, I’ve worked on numerous projects that have grown very quickly but have lacked a structured approach to development. The code is grown somewhat organically, being used in ways never imagined and bent into shapes it was never supposed to make. In the very worst cases, it ended up with Spaghetti Code that is self-obfuscating and unintelligible (even to the original author).

To help prevent this situation, teams need to embrace processes to help introduce structure and discipline to software development. This can be daunting to some teams at first, but it’s a far more productive and appealing than a team that is constantly under-pressure with various fire-fighting activities.

Kent Beck has said previously that he is not a great programmer but just a pretty good programmer with great habits. I’m a firm believer in that statement, and I think that any team that doesn’t try to develop those habits is starting from a weakened position. I’ve worked on both projects that embraced change and tried to improve their development process and also those that didn’t. I would sooner work on the former type of project every time.

With that in mind, there are very simple changes you can make to a project to help foster better habits. None of these techniques is new or groundbreaking (at least anymore), but I am still amazed at the number of projects that still don’t embrace them. There is plenty of information on these techniques out there, so there is no excuse if you want to do some more reading.
Read the rest of this entry »