On their blog engineers of logistics service Shyp (defunct) talk about how they improved the turn-around times of their JavaScript test suite by an order of magnitude of 3 (i.e. 1000x), which is no small achievement.
In a modern software development process continuous integration and continuous deployment play a vital role. These ensure that your software always is in an deliverable, tested state and ideally is deployed to production systems in a timely manner when a change has been made. This process requires sound and thorough unit and acceptance testing suites that run both on the developer’s machine after a code change and on a continuous integration server once that chance has been committed to the source code repository. Running these test suites on a development machine poses a few problems of its own though, the most important of which is that as long as the test suite runs the development machine and by extension its user – the developer – is blocked from continuing to code. This in turn leads to one of the most dreaded and expensive cost factors in software development: Distraction and context switching. Focus – otherwise known as flow is one of the most important contributing factors to an efficient development process. Losing this focus, being yanked out of the flow by the abundant distractions available or by having to switch context because of having to wait for the development machine to complete a task leads to a steep decline in developer productivity: The time to get back into the flow according to most sources is 10-15 minutes. Given that your test suites should be run as frequently as possible to make sure you didn’t break anything while introducing a change this creates a lot of waste that’s multiplied by the number of developers working on your product.
While in my opinion this should be reason enough for optimising your test suite as much as possible and making sure your tests run in a matter of mere seconds, the Shyp engineers offer a few other convincing causes for improving your testing turn-around times:
- Deployments get faster.
- Fast tests lead to better code.
So, there’s ample reason to make fast test suites an essential requirement for your software product. In their blog post Shyp give both useful tips as to how to measure time and performance improvements (using the UNIX tool ts) and suggest a variety of specific approaches for improving JavaScript test suite run times in particular. Those approaches involve not only general measures such as avoiding shared folders on VMs or network file systems during tests and faster database clean-ups but also improvements that are particular to their Node.js– / Sails.js-, / Mocha-based environment. A lot of those apply to most other JavaScript environments, too with Mocha being a widely-used JavaScript test framework.
Anyway, the blog post should allow you to get started with this important aspect of an efficient development process. From what I’ve seen in many a development shop chances are your own test suites are far from running at a speed that allows developers to work with almost zero turn-around time. So, this might be an area you might want to focus on for your next sprint (or whatever your project increment is). While the blog post mentioned focuses on JavaScript – which is particularly difficult to deal with in terms of efficient testing because of the many components and tools involved – there’s a wide range of approaches for improving test suite turn-around times in every language and environment.
If you need any further information or help with this (or other topics related to software development) please drop me a note.