- More time between checkin and a report of a failure
- Higher chance of Continuous Build containing multiple checkins, increasing the chance of an integration break and complicating rollback
- Fixing a build related to a checkin made much earlier decreases productivity, leading to a reduction in productivity
There are other ‘build’ times to be aware of. A long Checkin Gate build leads to an increased chance of someone else checking in before you, increasing the chance of an integration break when you do checkin. It also disrupts the developers normal flow – they cannot easily work on new code, so effectively have to down tools waiting for the Checkin Gate has finished. You also need to consider the time taken to run a single test – be it a small-scoped unit test, or a larger end to end test.
No matter what the build is, a long build interrupts programmer flow, decreasing focus, and therefore decreasing productivity.
Different Builds, Different Limits
As a team, you should decide on acceptable Build Time Limit for each ‘build’ – for example individual tests, Checkin Gates, and stages in your continuous build. You may even consider failing these builds if those time limits fail. Setting the Build Time Limit at the right level – and keeping it there – will help keep productivity high.
Different builds get run with different frequencies. The more often a build is run, the faster it needs to be. Experience suggests the following time limits:
- Single small-scoped unit test – sub-second
- End-to-end test – a few seconds
- Checkin Gate – 30 seconds to a couple of minutes at most
- Continuous Build – a handful of minutes
When your Continuous Build is part of a larger Build Pipeline, you may find it useful to set Build Time Limits for each stage in the pipeline. One might argue that enforcing Build Time Limits for each stage of a Build Pipeline – manual or automated – may be overkill, but having some reporting of when a limit is exceeded will help directly highlight bottlenecks in creating production deployable software.
Teams must take ownership of ensuring that the Build Time Limit is enforced. Further, they should always look for opportunities to reduce them further. Any decision to increase any Build Time Limit should be taken by the whole team – likewise any decrease in a Built Time Limit with decreases test coverage should be agreed with all. Everyone should be empowered however to look for quick wins.
Some teams find the need for a Build Tzar/Build Cop role – someone who is in charge of the health of the build. I consider such roles as being short term measures only, and should certainly be considered an anti-pattern if they exist for any length of time. At the extreme end of this spectrum is the dedicated build team. Empowering the whole team is key.
Making Things Faster
There are a number of ways of making individual tests fast, which will depend both on the nature of the technology being used and the way it is being used. Consider making a Checkin Gate fast using a Movable Checkin Gate. Long Continuous Build times can be mitigated through the use of a Chained Continuous Build, perhaps as part of a larger Build Pipeline.
You may also want to simply remove tests that are slow but provide little coverage. Often, it may even be the case that slow running tests represent a performance issue in the system itself.
Some teams have also shown significant speed improvements by using the right hardware – such as faster CPUs, RAM disks or SSD drives. However simply throwing hardware at the problem can help speed a Continuous Build up, but this does little to affect the build time on local development machines – a situation where your continuous build is faster than your local development build is the opposite of what you want.
For more concrete evidence on how build times can influence the productivity of teams, Graham Brook’s paper for Agile 2008, Team Pace – Keeping Build Times Down, details experiences of working with two different teams and the impact of long (and short) build times on the development team. Thanks also go to Graham for reviewing an earlier draft of this article.