TDD is not perfect - but it's still pretty good!

Late last week, on LinkedIn, I shared some initial thoughts on why software developers continue discussing (or having flame wars about) automated software testing, and TDD (Test-driven Development) in particular, which I’ll center the post around.

In case you missed it (though I recommend checking out the post), I suggested that there were likely a number of very valid reasons why some developers continue to question the validity or necessity of automated software testing. However, I also suggested that it’s worth using, learning, and forming your own opinion, rather than just accepting any one person’s opinion as to its value.

After all, I finished the post with:

Am I encouraging you to try? Definitely!

And, it makes sense that I’d be enthusiastic about TDD, as it’s helped me – over the course of 20+ years. The more I’ve learned about it, and software testing more broadly, and the better I apply that knowledge, that I’ve written better and better software.

To be fair, I started off from humble (read: poor) beginnings, way back around 2001. If we don’t know each other that well, or you didn’t know me back around that time, I turned to TDD after having a dire experience writing an Excel VBA application, one which went horribly wrong.

Image sourced from https://corporatefinanceinstitute.com/resources/excel/excel-vba/.

To provide some more context and background, I was working for a small company in the heart of Sydney (Australia) and was tasked with writing the app. I think it connected to a Microsoft Access “database”.

I don’t remember too much about the experience really – but I remember how it ended! I was super-frustrated as (at least to me) the application was not working as I had written it, with an ever-increasing number of “random” bugs which I was struggling to track down. The boss expected it to be finished ASAP, something that I just couldn’t deliver upon.

Naturally, I don’t have the code with me. And I wouldn’t want to see it and be embarrassed by it. However, I surmise that there were a number of logic bugs in the application which were causing the “random” bugs. I also surmise that I could have caught most, if not all of them if I’d been practicing TDD when writing the application.

2001 wasn’t that long after its introduction. So I don’t know if there were any unit testing frameworks for VB6 or VBA back then. But, a quick bit of searching uncovered SimplyVBUnit. I’ve not looked into the project, but it might have been available back in 2001. Though, since it’s modelled after NUnit, which was released in January of 2013, perhaps not.

Regardless, after writing tests for the better part of 20 years now, I’m confident that my experience would have been a lot better back then if I’d been writing tests, even if I didn’t know very much.

TDD isn’t perfect

Despite this enthusiasm, in this post I want to share that, even though I’m an enthusiastic supporter of software testing, and TDD in particular, I accept that it’s not perfect. Well, “perfect” isn’t quite the right adjective. Rather, I don’t propose that just by using it, your code will magically work properly, handling all edge cases, and that there’s nothing else you’ll need to do to write highly maintainable, easy to read, performant code.

I want to talk about one particular way in which tests can pass but the code doesn’t work. So, you feel a false sense of confidence in your code. Specifically, it’s where you write broken tests because they test the wrong thing.

As an example of what I’m referring to, recently, I’ve been building a small PHP-based web application that forms the basis of an upcoming Twilio tutorial. The application shows how to use Twilio’s Verify API to provide a more secure login process, before the user can then upload PNG and JPEG images to the local filesystem.

If you’re not familiar with Twilio’s Verify API, don’t worry. I’ll do my best to explain it enough for it to make sense.

The login process consists of two parts. It starts off with a form where the user submits their username, which you can see above. If the username is in the list of available users (currently just a plain old PHP array) the application retrieves the phone number linked to their username and sends a verification code by SMS to that number using Twilio’s Verify API.

After the code is sent, the user is redirected (via a 302 redirect) to the second step in the login process. Here, they enter the code that they received on their phone. What’s not immediately obvious, is that the form also stores their username in a hidden field. This is mandatory, as the user’s phone number is required to validate the code that they were sent.

The verification form checks if the username is available in a request header. If it’s not available, the user is redirected back to the login form. If it is, the verification form loads, ready for the user to supply the code that they received, which you can see in the screenshot below.

When the form is submitted, another call is made to Twilio’s Verify API to verify if the code that they submitted in the form was the one that they were sent via SMS. If so, then they have now logged in and can begin uploading PNG and JPEG images. If not, they’re redirected back to the verification form.

There’s a bit more to it, but that’s the general idea.

Now, the important part, for this tutorial, is how I persisted the user’s username between the login and verification forms. I initially – wrongly – thought that I could store the submitted username in a header in the redirect response from the login form to the verify form, and then use that header in the route that was redirected to.

Here’s, roughly, the code in question:

if ($verificationInstance->status === 'pending') {
    return new RedirectResponse(
        uri: '/verify',
        headers: [
            'USERNAME' => $formData['username'],
        ]
    );
}

If you’re not familiar with the RedirectResponse, it emits a Location header to the URI specified by the uri parameter, and sends a custom header, named USERNAME with the value of the username field submitted in the login form.

In my head, this was a simple way of persisting the username between the forms. Except, it was never going to work! If you’re paying attention, and you’ve got a clear understanding of requests and responses in HTTP, you’ll know why.

In my misplaced confidence, I began implementing the functionality, letting tests drive how I did it. When I ran the tests, after writing the applicable code, the tests were green. So, based on that, I had a false sense of confidence that the application would work properly.

However, when I manually ran it, it didn’t. 😔 I would submit the login form with a valid username and a verification code would be sent to my phone via SMS. Then, I would be redirected to the verify form. However, I would then be redirected back to the login form.

Why? There was no USERNAME header containing the username in the request to the verify form. What gives?

To cut a long story short, there was no way that a response header was ever going to be available to a subsequent request. Why? Take a look at the start of RFC 7231’s introduction:

Each Hypertext Transfer Protocol (HTTP) message is either a request or a response. A server listens on a connection for a request, parses each message received, interprets the message semantics in relation to the identified request target, and responds to that request with one or more response messages. A client constructs request messages to communicate specific intentions, examines received responses to see if the intentions were carried out, and determines how to interpret the results.

As it says in the first sentence:

Each…message is either a request or a response.

So, after the client sends the response, regardless of what’s in it, the server’s job is done. Now, it’s waiting for the next request from the client – which is a completely separate process!

Given that, the username that I set in the custom redirect response header was never going to be available to the following request. As a result, the user would only ever be redirected back to the login form.

The tests passed, but they were wrong!

Before you tell me “I told you so” or “TDD is broken”, bear with me. And keep this in mind:

Tests are a tool!

Just like linting, static analysis, and code style guides (among so many other things), they’re tools to help you write good, clean, working, and maintainable code. However, you decide when, where, and how to use them. Because of that, and that we’re all fallible, you can make mistakes when using them, such as this one that I made.

There’s nothing inherently wrong with TDD, rather with my application of it. A tool doesn’t know what you want to achieve, and only within very limited bounds knows if what it’s being asked to do is distinctly right or wrong. You’re still the one making the decisions.

They are there to help you, but they’ll never hold your hand for you. If you employ them in the wrong way – and you blindly trust them – then they’ll do what you ask, and in the process lead you down a proverbial garden path where you believe your application is fine, when it’s not.

That’s why it’s important to see automated testing, in particular TDD, more holistically rather than a box you tick once you have a test suite of some scope. Instead of asking if you have enough tests, see it as a process that helps you think through the problem that you’re trying to solve in a way that is verifiable.

It frustrates me when people reduce TDD to either: “Do I have tests?” or “How close are we to 100% test coverage?”. The tests themselves are an implementation detail, an outcome of following the process.

Yes, when done well, they can add to a project’s documentation and help developers know how the code is expected to work. But, like so many other software development approaches, think Agile and Scrum, people – often non-developers – focus on things that you do, rather than the bigger picture.

I think this is a reflection of us as a society; which is definitely a topic for another day.

TDD is but a tool you can use – but it’s a pretty good one

It’s too easy to start writing code that becomes a big ball of proverbial mud without techniques such as TDD. Why? Because when writing the code as you go you’re often not stopping to consider different possibilities and eventualities.

However, it’s much harder to do this when you write code guided by tests. Instead of thoughts such as “I have to get this done”, “Oh, yeah, that’s how I’ll do it”, or “What if I did it this way?”, you force yourself to answer questions such as: “How would I test that?”, “What if that input was not available, or was set to something unexpected?”.

I feel, and it has been my experience, that you think in a much broader, more comprehensive way when you use TDD.

This hasn’t been an exhaustive discussion of why automated testing is a solid addition to any software development toolset, however, I hope that it’s helped you see that it is a good addition, one that improves the quality of what you write, regardless of the size of the project.