You are here

Gated check-in system tightens DevOps feedback loops

public://pictures/18150dc.jpg
Eitan Schichmanter, Senior DevOps Architect and Leader, eBay

In DevOps, the development lifecycle usually involves developers writing their code, testing it locally on their machines, and then committing the code to a source control management system. This causes an automatic build to start, and the resulting code is evaluated to guarantee its stability, integrity, and quality.

As soon as a developer commits his or her code to the release branch, it becomes part of the release code. If the ensuing build fails due to that commit, the release branch's integrity is compromised. I crunched the numbers on the products I work with, and I found that an alarming 15 to 30 percent of builds on the release branch fail for various reasons (code, test, and environment issues), all contributing to the poor stability of the release branch.

[Also see: Feedback loops: The confluence of DevOps and big data]

The direct consequence is a severe decline in the developers' velocity, which diminishes developer trust in the build system—catastrophic for DevOps. Whenever we had a code issue, a storm of "Who's on it?" emails ensued as soon as the code was committed, as the break spread virally throughout all the development teams.

My team wasn't prepared to work like this any longer. We needed a solution.

[ Get up to speed on quality-driven development with TechBeacon's new guide. Plus: Download the World Quality Report 2019-20 for lessons from leading organizations. ]

A new way: The gated check-in system

Our demands were quite simple: Stop the code from being committed to the release branch, verify it offline on a copy of the most recent release code, and pass it back only if the verification is successful. This type of solution is called gated check-in, and it comes in several different flavors and implementations.

You can find existing gated check-in solutions out there: Gerrit, Microsoft's Team Foundation Server (TFS); Zuul; and Electric Cloud's Preflight solution are all examples, and each has its pros and cons. For example, Gerrit is widely used in the open source community, as it integrates into Git and Jenkins, and Microsoft's TFS is built in with the entire development environment.

The required characteristics and features of a gated check-in system depend on the current development flow, including but not limited to the following:

  • Non-intrusive (it doesn't change the developer's workflow) and able to run on the build server, not on the developers' machines
  • Easy for developers and DevOps to use and maintain
  • Stand-alone and development-stack agnostic
  • Lightweight and simple to enhance
  • Efficient parallel commits algorithm (addressing issues such as how to tackle concurrent commits while verification is already running)

My team and I developed a solution that we call Verigreen (verify your release code—keep it green) to address all the previous requirements. We've released it as an open source project on GitHub.

[ Learn how to apply DevOps principles to succeed with your SAP modernization in TechBeacon's new guide. Plus: Get the SAP HANA migration white paper. ]

[ Get up to speed with TechBeacon's Guide to Software Test Automation. Plus: Get the Buyer’s Guide for Selecting Software Test Automation Tools ]

Article Tags