Last year, more than half of companies surveyed by XebiaLabs said they were looking to implement continuous delivery—more than those looking to implement agile and DevOps. The motivation for this, according to cloud-based Jenkins service vendor CloudBees, is that "continuous delivery allows you to deliver new software (business value) and improve existing software faster, with lower risk."
Should the skeptic buy this claim? Let's examine continuous integration (CI) and continuous delivery (CD) in detail and figure out the real costs and benefits.
Some definitions
According to CI/CD/testing software and consulting vendor ThoughtWorks, CI "is a development practice that requires developers to integrate code into a shared repository several times a day. Each check-in is then verified by an automated build, allowing teams to detect problems early." ThoughtWorks chief scientist Martin Fowler adds that "Continuous integration doesn't get rid of bugs, but it does make them dramatically easier to find and remove."
According to the same vendor, "Continuous delivery is the natural extension of continuous integration: an approach in which teams ensure that every change to the system is releasable, and that we can release any version at the push of a button."
Why do I quote Fowler and ThoughtWorks on CI and CD? Because they wrote the seminal articles and books on the subject.
What does it take to implement CI and CD?
Some CI and CD vendors claim that all it takes for your company to adopt these development practices is software and a little setup. If you believe that, I have a bridge to sell you.
What it really takes for most companies to implement CI and CD is to completely change their software development culture, adapt their organization, change their workflow, automate the bulk of their testing, and either install significant infrastructure or learn to love the cloud. None of that is impossible, but none of that is easy.
The technical parts are easier than the organizational and cultural parts of this shift. Putting your repositories on Git, for example, may involve a painful conversion process. But the hard part is training your developers not only to check in their work multiple times a day but also to pull from the repository before each coding session, promptly merge any conflicts (when it's their responsibility), and write good tests that can be incorporated into the automatic test suites. It's easy to use Jenkins, for example, to trigger builds on check-ins and send out reports to the appropriate developers; it's harder to make the developers pay attention to the reports in a timely fashion and shelve their new development to fix their older mistakes.
Note that "we can release any version at the push of a button" isn't always a good thing. Suppose you need to check in a feature that isn't ready for end users, perhaps because it relies on a dependency that's still a stub or because management is saving it until after a public announcement? It's nice to get it integrated and tested early, but you have to come up with a way to hide it from end users until it's actually ready.
Without CD, you could just isolate the feature to a branch and merge later. With CD, you're not supposed to do that. One way of handling this situation is to implement a feature toggle that will hide the user interface or possibly even skip over the relevant code until it's turned on again. But a better way to manage this is to come up with a staged integration plan that is always shippable.
The fly in the ointment
To summarize, it's easy to generate feedback to the developers that says "the build is broken," but it's hard to get them to stop work and fix the build. Yegor Bugayenko has discussed this problem in depth and concluded that CI is dead.
Why? In the book, Continuous Delivery: Reliable Software Releases through Build, Test, and Deployment Automation, Jez Humble and David Farley of ThoughtWorks wrote, "Crucially, if the build fails, the development team stops whatever they are doing and fixes the problem immediately." In Bugayenko's experience—and mine—that isn't what actually happens. In real life, everybody ignores the build report until they have spare time to fix it, so the product is typically not shippable for a matter of days until the team gets around to it.
Bugayenko claims that the solution to this problem is to keep the master build from ever breaking by using a read-only master branch. In this workflow, the development branch goes to a script that merges, tests, and then commits to master if and only if all the tests pass. If any test fails, the branch is rejected, and you get a message to tell you what tests failed. Then you can fix your code and try the script again.
Other people claim that the answer is a gated check-in, as implemented in Team Foundation Server. That works well if the product is small and the build farm is big, but it doesn't work so well if a build can take hours. Still, other people will automatically revert any commit that breaks tests. And some require a code review before any patch set can be committed to the CI server.
Some skeptical conclusions
Proponents of CI/CD claim benefits of accelerated time to market, faster customer feedback, improved developer productivity, reduced technical risk, improved quality, and improved customer satisfaction.
I don't question any of those claims.
What I do question are the costs of making the transition to CI and CD, the difficulty of changing the company's software development culture, and the practical difficulty of always maintaining a shippable product. If you're realistic about all of these and plan your remediation strategies well, then you can benefit from continuous integration and delivery.
As for the answer to life, the universe, and everything, we all know it isn't continuous integration: It's 42.
Image source: Flickr
Keep learning
Take a deep dive into the state of quality with TechBeacon's Guide. Plus: Download the free World Quality Report 2022-23.
Put performance engineering into practice with these top 10 performance engineering techniques that work.
Find to tools you need with TechBeacon's Buyer's Guide for Selecting Software Test Automation Tools.
Discover best practices for reducing software defects with TechBeacon's Guide.
- Take your testing career to the next level. TechBeacon's Careers Topic Center provides expert advice to prepare you for your next move.
