Would it be sacrilege to decrease the amount of code testing? Surprisingly, there is ample data showing benefits from less testing. A typical software testing suite has these drawbacks:
- Takes too long to execute
- Tests code that hasn’t changed
- Runs single-threaded
- Costs companies countless hours and many dollars
Software testing proves that software operates according to functional and non-functional specifications. It also provides feedback to developers about the quality of their code changes.
It’s considered best practice to keep code changes small in scale, reducing the scope and breadth of change. Most continuous integration automation, however, kicks off comprehensive testing suites that test thousands or millions of lines of code, even when few lines of code changed.
Automated test suites can take 12 hours to execute and thus might run only once per day. For others, automated testing can take one to two hours and execute several times a day. In both cases, developers must wait for test feedback, even for simple bug fixes and cosmetic updates.
Tip 1: Your test suite should complete in ten minutes
Ideally, software developers write and deliver code for 40 hours a week, but meetings and normal interruptions often push actual development below 30 hours per week.
One cause: developers spend too much time waiting. Once they’ve written the code, they wait for it to be reviewed; they wait for code to be deployed; then they wait for functional and quality assurance test results. Finally, they wait for validation, and integration tests.
Getting code to production takes too long. Sometimes developers wait a day, or more, between each step of the coding process. Even when they are delayed “only” an hour or two before getting feedback on code changes, productivity drops sharply.
While waiting, developers may be tasked with low-effort bug fixes. There is a cost to this “context switching,” which interrupts the intense, creative focus required for analytical problem-solving. Getting into that frame of mind takes time and energy. For many, context switching sabotages that productive mental state and decreases motivation.
Whether twiddling thumbs or context switching for an hour, developers are being highly paid to wait or create less value. The solution: Significantly speed up feedback on their work.
We observe that for many customers—10 minutes is a minimally acceptable threshold for waiting. It’s short enough to get a cup of coffee, stretch, take a bio break, or check in with a colleague—knowing they’ll have test results when that chat is over.
Tip 2: Reducing test suite execution time by 90% is doable
Many are surprised to learn that it’s quite feasible to reduce test suite execution times by up to 90%. Take one of our key customers as an example. Their test suites took between 90 and 200 minutes until recently. Now they usually take 10 or 11 minutes.
How can this be? Why aren’t all test suites faster?
In the early 2000s, most web apps were written as “monoliths.” New functions were bolted onto an ever-growing codebase. Yes, there may have been classes and objects and other performance optimizations, but most Fortune 500 companies on the web were building “macroservices.” These monolithic codebases grew to millions of lines of code with little abstraction between major pieces of the app(s).
Though the discipline of application architecture has moved onto cloud computing, microservices, and containerization paradigms, testing has not evolved in step. Testing is one of the most undervalued (or ignored) parts of the software development lifecycle. Most test suites are built like the monoliths of yesteryear—unnecessarily, because testing technology has advanced.
Test suites can be multi-threaded, but often are not. They can selectively run just the tests that are relevant to code changes, but often the entire suite is run arbitrarily. They can be architected for intelligence, acceleration, and reporting, but rarely are.
A modern software approach to testing can lead to modern testing results. It will not take years of refactoring and management; in fact, it’s not very complicated.
Tip 3: Enterprise-level test acceleration can be implemented in one sprint
The power of large enterprises can be difficult to harness. It’s no small feat to corral multiple departments with disparate priorities, some of them directly in conflict. Despite high levels of goodwill and a determination to succeed, projects inevitably take longer and cost more than initially estimated.
Why does this happen? Because large implementations require behavioral changes, infrastructural buildouts, tooling updates, and more.
Development teams need to leverage modern tools that impose no change on their workflows, and allow them to use the same commands, commit processes, and test procedures. What changes is how long they wait for a test suite to complete. In a best-case scenario, preparing a test should only entail adding a single gem or package to a dependency list.
Tip 4: Faster testing keep developers productive, happy, and on board
Developer roles are more important and in-demand than ever. A December 2020 jobs report by CompTIA notes that developer jobs accounted for 62,900 (16%) of the 391,000 new tech roles hitting the market. Tech companies live and die by the quality and retention of their engineering staff and right now, there’s more demand than supply.
When developers can’t actually develop, they lose motivation and enthusiasm for their work. Waiting for feedback and busywork produces frustration. That’s when they start looking for another job. Conversely, the more time developers spend producing code, the more they enjoy it, and the longer you retain them.
It’s far easier to hold onto developers who have the tools to be productive, enjoy their work, and feel motivated.
Tip 5: Test acceleration can drive the next leap in developer productivity
In the past 20 years, developer productivity has been boosted by microservices architectures, containerization, cloud infrastructure, and serverless. Our ability to build and run faster, more ephemeral code advanced at a rapid clip. Our ability to test did not.
It’s now possible to change that, and – in many organizations – save thousands of dollars weekly. Now that we have modernized infrastructure, coding paradigms, and scaling, it’s time to address obstacles that developers face in code creation.
Test acceleration can drive major advances in developer productivity. Companies should examine the status quo in their development processes; should test suites take hours or just a few minutes? Should developers be context switching or allowed to keep focused on coding?
For those who believe test suites should take just minutes, that developers should not be forced to context switch, and that code delivery times should be faster, we can demonstrate how that is achievable.
About the Author:
Yves Junqueira, CEO and Co-Founder, YourBase
