Why write tests?

The world of “software testing” is quite a confusing one and it takes several years to understand what’s going on and how to do things “right”. In no particular order:

• Developers use different words for different types of tests, but also for different types of test doubles.
• Developers are looking to achieve widely varying goals by writing tests.
• Developers have divergent views on the cost of writing and maintaining a test suite.
• Developers don’t agree on when to test software (before, during, after writing the code).

If you’re a programmer it’s your inherent responsibility to prove that the production software you deliver:

• … is functioning correctly (now and forever)
• … provides the functionality (exactly, and only) that was requested

I don’t think anybody would not agree with this. So taking this as a given, I’d like to discuss the above points in this article and its sequel.

Test terminology

As always when it comes to naming concepts, it’s very important to do it in a consistent and ubiquitous way. It’s of the utmost importance to establish a common understanding within your team. I advise you to pick a standard, make sure everyone understands it, and finally enforce it within your team. Point people to misused test terms.

For test types and their names, I recommend picking the naming convention proposed by Steve Freeman and Nat Pryce in their book Growing Object-Oriented Software Guided by Tests. Nat Pryce has written a useful blog post about Visualising Test Terminology.

A practical suggestion is to mark certain tests in your project as acceptance, integration, unit tests, etc. This should make it easier to run only specific groups of tests in different stages of your build process.

For test doubles and their names, I recommend sticking to the “standard” list of test doubles, as described in XUnit Test Patterns by Gerard Meszaros. I can also recommend the more conversational article The Little Mocker by Uncle Bob.

Goals of testing

Software testing has often been advocated by screaming: “Write tests!” as well as instilling a sense of shame into developers who don’t write them. I think it requires a more subtle approach as well as some explanation. Why write tests?

Verifying correctness will be much faster and less troublesome

Writing production code without any accompanying test code, the only way you can be sure that your code is correct is by exercising the program: running it, clicking your way to the page or screen you’ve been working on and verifying if everything works. You may need to log in, maybe even wait for a text message with some secret code, open the menu, select an item, wait for the page to load, wait for the application to connect to some external API (which may need to be configured too before you can use it), and so on. This will soon lead you to cut corners.

• You won’t try all the possible scenarios anymore.
• You will hit random keyboard keys to fill in forms, etc.
• You may skip the “exploratory testing phase” entirely, re-read your modifications and assume they will work properly once released.

Manual exploratory testing is a huge pain. Of course, your customer is going to do it, but you shouldn’t rely on that. They will cut corners too, leaving the discovery of a potential problem to the actual users. More importantly, you need a better way to feel secure about your work. Now remember our initial assumption: you have to prove that the software you deliver functions correctly. This is why you have to create automated tests for your application.

By specifying expected behavior you’ll know when you’re ready

As a developer I’ve created many applications for which expected behavior wasn’t properly defined. So me and the other team members resorted to guesswork, writing a lot of code. We developed features that were “probably going to be needed”. We set things up in a pretty generic way, “just in case”.

Once you start writing tests, and more specifically, when you start specifying your application’s behavior from the outside perspective (known as acceptance testing, or “specification by example”), you will find that you write just the amount of production code necessary to make the application behave in the expected way.

You can use the written specifications as an instrument for customer collaboration: they can be written together with a stakeholder or when a developer writes them on their own, they can be used to verify that the developer understands what should be built and is going to work on the right thing.

By specifying behavior and making those specifications executable and verifiable, you satisfy the second part of our initial assumption: that the software you deliver provides the functionality that was requested; exactly that, and nothing more than that. This approach is called BDD - Behavior-Driven Development.

If you want to learn more about BDD and specifying behavior, you could start out by reading the article Introducing BDD by Dan North, or the book Bridging the Communication Gap by Gojko Adzic.

You will feel a lot safer to make changes

Once you have specifications in place (I do think we might be better off using the term “specifications” instead of “tests”), it will be a lot safer to make changes (including refactorings) to your application. By running the tests you can be certain that changes in one part of the application didn’t cause things to break in some other part. Having an extensive test suite will thus let you catch regressions (hence the term “regression testing”). This saves a lot of time manually testing an application and is therefore much cheaper.

Feeling safe to make changes and being able to do so quickly is very important:

• It will make your work much more enjoyable and less messy.
• It will make you or your company much more valuable for your customer.

The software will have a better design

If you write tests after writing the production code that those tests are for, you’ll discover many aspects of production code that make it hard to produce a test for it. Once you feel the pain of untestable production code, you will start looking for ways to improve the design of your code. You start injecting dependencies instead of invoking them in a static, global way. You start depending on interfaces instead of classes to allow dependencies to be swapped out. And you’ll expose hidden dependencies on, for example, a running database or the system clock.

I find that improving the design of your code (or your application architecture for that matter) is good in itself, and will prove to be helpful in general. And once you’re starting to provide automated tests for your code, it will be much easier to do so if the code is well designed. But once you know design principles like the SOLID principles, and you apply them naturally, you might as well start writing tests before writing the production code. This is known as TDD: test-driven development.

The software will end up being maintainable

The combined power of a better design (that comes from making software testable) and the exact specifications (of the application’s expected behavior), the purpose of the application should be very clear by reading through the tests and specifications. This will make the application maintainable, a very important trait for projects that should live longer than a couple of months.

For me, this last point is related to some interesting statements. One that I often say to developers and that was inspired by Michael Feather’s definition of legacy code as “code without tests”:

If you deliver code without tests, it’s legacy code.

It’s not maintainable, which means it’s going to deteriorate very quickly.

A related statement is that:

If you had to choose between throwing away test code or production code, then choose production code.

This one is based on a thought experiment posed by Uncle Bob (couldn’t find the source, sorry). If you throw away your production code, you could relatively easily rewrite the production code based on the tests/specifications. If you throw away your test code, you have turned your code into legacy code and gravely endangered the future of your project.

Up next

What’s left for us to discuss are the economics of tests and the right moment to write tests. We’ll cover these topics in another post.