Tuesday, March 5, 2013

Unstable builds in Jenkins

We use Jenkins for our continuous integration server, and I've never really seen the need for it to have an "unstable" build status. I actually think this build status does more harm than good. I've found that unstable can be treated in one of two ways - either it is treated like a build failure and fixed immediately or it is treated as not important and ignored. Either way, I think it loses its meaning.

It's also a bit unclear as to what "unstable" actually means. Some coverage plugins for example will set the build to unstable if the coverage level falls below a certain threshold. In this case, if coverage is too low, I wouldn't feel comfortable producing a build that is production ready, so I would fail the build. It's also possible to configure Jenkins to mark a build as unstable on test failures, instead of failing the build - I'm not even sure why that option exists.

I believe the build should only have two states - passed or failed. Whenever I start a new Jenkins project, I always configure any of the plugins I'm using to not use the unstable state (configuration varies from plugin to plugin). I've found keeping out the "unstable" state has led to healthier builds overall.