As companies strengthen their DevSecOps practices, there's a pressing need for quality-driven leaders to drive "shift left" testing. In fact, QA is evolving to become an integral part of the development team and delivery lifecycle from end to end - including security, according to the World Quality Report 2020-21. Your entire organization benefits from having testers involved earlier in the development process.
There are other advantages, too. Test teams gain a better understanding of the full system under test and participate in improving user stories and design. Automation developers get the time they need to stay on pace with application development, and test cases are more thorough.
Even so, these activities traditionally require a dependency on upstream deliverables that keep testers in a "follow" mode instead of allowing them to lead.
Here are three practices testers can use to change their approach, and lead a quality-driven transformation toward DevSecOps (aka secure DevOps).
Share your test strategy
You can have a quality product only if the feature provides value to the end user. Product owners have the challenging role of understanding the voice of the customer and turning those ideas into testable, small, independent user stories. They may not scream for it, but they can use the help of a QA professional who shares the mission of deriving high value from each feature.
As defined in acceptance test-driven development (ATDD), the development of acceptance criteria should be a team activity in which the tester adds a unique perspective. Expanding acceptance to cover essential conditions will only help developers build the right product the first time.
Even with the increased focus on test-driven development, my QualityWorks team and I have found incredible success in sharing the test strategy with developers before coding begins.
Either in sprint planning or shortly after, I set aside a few minutes with the story developer to discuss my planned test strategy and often provide a checklist to consult while coding. It's almost like the teacher giving the student the answers to the exam.
We both start the sprint stating that we want to deliver an A+ product (zero-defect story) and more often than not we meet that goal. It also helps developers estimate tasks more completely, ensuring that they complete stories with testing time in mind.
Define quality in your definition of 'done'
Depending on your organization's agile maturity, the execution of the definition of "done" could range from "does not exist" to "religiously followed." In my opinion, the definition of done (DoD) is just another quality control. The tester should promote continuous improvement of the DoD, especially if the list is stagnant.
For example, you shouldn’t have metrics such as a 90% functional test pass rate that exists for years. This number should increase and decrease as the organization learns what works.
When the DoD is developed by leaders without validating the feasibility of enforcement, there are bound to be problems. I've experienced a management edict such as, "All user stories must have 100% code coverage." Unfortunately, at the time of the decree, there was no unit test framework or code coverage tool integrated into the development process. It inevitably led to stories being blocked, and created a culture of DoD steps being ignored.
As a hero for quality, use these opportunities to create and promote the prioritization of stories for your technical debt that make DoD controls easier for the team to accomplish.
If there is a long backlog of technical debt, I recommend teams start the DoD with minimal viable criteria as opposed to building a full wish list of controls that are not feasible within a sprint.
For the more mature teams that are meeting DoD criteria consistently, begin to add the definition of "secure" for your DevOps into the DoD list at a pace that is realistic to accomplish.
Measure, measure, measure
Most tech organizations are likely to link their product delivery challenges to the lack of technical expertise or resources. However, experience shows that the root cause of the low return on technology investment often lies in the inability to manage enterprise change from both an organizational and strategic perspective.
Metrics are foundational to the continuous improvement of your DevOps practices. They provide exposure to team performance, including successes, blockages, and inefficiencies.
Testers have a significant role to play in collecting and advocating for the collection of some key metrics that can add value and help to inform the organization's DevOps efforts. Value is subjective to your company goals, product, and people.
Key metrics to collect
Here are some important metrics that testers should advocate collecting. (The ease of collection and impact of metrics are based on my experience coaching agile teams.)
- Direct user feedback, capturing satisfaction and engagement of real users (hard to collect, but high value)
- The velocity of story delivery (easy to collect, medium value)
- Code coverage (easy to collect, medium value)
- Number of defects reported by manual QA (easy to collect, medium value)
- Number of defects reported by automation (medium difficulty, high value)
- The velocity of releases (easy to collect, medium value)
- Number of customer-reported defects (hard to collect, high value)
- Costs of delivery (hard to collect, high value)
- Team satisfaction (medium difficulty, high value)
The ease of tracking or managing these measures very much depends on your organization's infrastructure and collection systems. If you're not sure where to begin, I'd suggest collaborating with your team to select a starter list of measures.
You can then create technical debt stories to define initial benchmark collection criteria and decide on how to simplify the ongoing tracking. By iteratively increasing the measures tracked, you can use them to drive healthy competitions and analysis in retrospectives.
Small steps, big impact
Regardless of where you are on your DevSecOps journey, these are easy-to-implement steps that require minimal effort but will have a significant impact on product delivery.
If you're a test expert reading this and would like to be more integral to any DevOps transformation process, I’d suggest you start small. Speak with your product manager about helping translate end-user goals to acceptance criteria to ensure that no critical features are missed. As a tester, paying great attention to details is a part of your DNA and can be a great asset in the initial stages of product development.
Don’t hesitate to share your test strategy. If developers and testers are aligned on what the quality of the end product should look like, it reduces the likelihood that there will be a bunch of inconsistencies and rework for both parties.
Consider the DoD as another area where QA has an opportunity to add significant value. DoDs are essentially another method of quality control. Testers should be the promoters of clear, doable, and continuously improved DoDs that will improve the accuracy and efficiency of the delivery process.
Take the lead
Being able to think through the effects of decisions and particular initiatives is a critical element in the process of delivering great products that users love and value. Testers are almost hardwired as a part of their job function to think through how different actions will result in different outcomes.
When it is translated into the collection of metrics, this area of expertise can prove indispensable to the company's overall initiative to improve its software delivery process. In an ideal process, understanding how important metrics such as customer feedback will be collected should precede the development of any product. Testers should advocate for the continuous collection and use of data to inform project and strategic decisions.
The recommendations offered here are just the tip of the iceberg on the value of taking a quality-driven approach to DevSecOps transformation. Should you take a minute to step back and assess your journey, you will discover a myriad of opportunities where quality could be the driver for a more efficient and successful software delivery process.
