Testing
Even among friends, testing leads to divisive conversations. It's a complex topic that, done poorly, breeds the exact carelessness it was meant to prevent. Engineers who ignore tests aren't lazy; they're responding rationally to a suite that has stopped being reliable, convenient, or effective. This chapter outlines the levers we can pull to improve how we test our software.
Automation builds the foundation for any successful testing strategy. To catch errors, we run tests. To catch errors as early as possible, we run tests often. Every change to our code base warrants assessment, and we execute the same tests hundreds (or thousands) of times a week. Far exceeding any scope we could realistically cover with manual labour.
We test throughout the life of a code change with varying suites of automated tests. For every step, we provide context as to why that particular constraint exists to make it feel less arbitrary. People are less inclined to push back against rules when there is a clear reason behind them. With trunk-based development, we ensure the integrity of changes at the following times:
- On our developers' machine during development
- On merge requests (Pre-merge)
- After a merge into main (Post-merge)
- Periodically on the main branch
- When creating a release candidate (Pre-release)
- Against the live version of our software (Post-release)
We run our tests as often and as early as feasible. The sooner we detect an error, the less money and time we spend correcting it. We dive into the scope of different test suites in the two sub-chapters.
On Test-Driven Development
When it comes to testing, the engineering community defaults to a popular strategy called Test-Driven Development (TDD). Popular as in well-known, not necessarily well-liked, widespread sources define TDD as the practice of writing tests before writing the source code. An implementation is considered complete once it passes all tests and edge cases. Practically, I consider this definition lackadaisical.
Instead of declaring an order of implementation tasks, a practical definition of TDD prioritizes code testability above all else. We design our system, classes, methods, and API for writing uncomplicated, performant, and resilient tests against them. We consider internal data accessibility, how we inject dependencies into implementations, and how to set up and tear down states of our software.
Our code encourages painless use of test doubles and computes without file i/o or network i/o operations. Any method that operates on data ordinarily loaded from disk, offers the capability of injecting mock data for running tests. It's difficult (and pointless) to write tests for these implementations before the source code is mature enough to run tests against it.
Beyond functional testing, TDD encourages us to consider post-release monitoring and observability during development. Every form of telemetry demands computational resources. Planning these while writing the business logic allows us to minimize the runtime cost of telemetry. We decide which metrics we care about and how to extract, store, and transmit these to our server with minimal impact on software operations.
Any engineer who went through the joy of adding tests or telemetry to legacy systems can share war stories of the task. Appending tests in hindsight involves ripping the code apart and introducing wrappers. The majority of testing-after-the-fact projects carry a high risk of introducing regressions in functionality and performance.
A systematic methodology-driven approach encourages us to document, plan, and tackle problems in small increments. TDD further ensures the developer of the feature owns the problem and the tests. When the complexity of testing increases, some companies defer the majority of testing to dedicated teams of QA engineers. These take over the testing and infrastructure provision for large-scale efforts including end-to-end tests involving UI, middleware, backend services, and database entries. A sound approach, provided this facilitating team does not write the actual tests.
It is reasonable to assign the selection, maintenance, or development of test tooling to dedicated teams. Some languages come with native testing tools, while others require frameworks and external dependencies to execute test code. It is not reasonable, in fact, we discourage moving the responsibility of writing and executing the tests away from the developer writing the functionality.
If an engineer hands off the quality control of their work with only a description of the expected behavior, our development velocity grinds to a halt and future regressions may not be caught until late into the development process. To reduce cycle times, it is imperative that the feature developers write and maintain the tests.
-
Good read? Unlock the rest of the chapter!
Engineering Collaboration is currently available as an Advanced Reading Copy for select readers.