When development and operations come together, it's not always clear where quality assurance (QA) testing fits. Nonetheless, organizations need to decide quickly, because the trend toward integrating software engineers and other IT professionals into a single cohesive team—DevOps—is gaining momentum.
Gartner predicts that DevOps will become a mainstream strategy in 25 percent of Global 2000 organizations by next year. While the allure of DevOps has led to the rapid adoption of this new mindset, it's important to stop and consider where testing fits into your operations. It's about more than just automation. Manual testing, and exploratory testing in particular, plays a critical role.
As the lines between development work and operational management blur and teams strive to create a deploy-on-demand framework, they often assume that testing will be automated. They may also assume, erroneously, that manual testing isn't as valuable or efficient. But automation can only cover so much ground. You still need the insight that exploratory testing offers. You also need QA professionals to identify what should and shouldn't be automated and to manage that process.
Automation is no panacea
Your team may want to handle more of the testing with a continuous integration approach, because it prizes the ability to run things over and over as quickly as possible. They may think there's no room or time for interventions or support. Automation is an obvious way to go, whether it's running smoke tests; developing backup, restoration, and rollback scripts; handling deployment; and verifying data integrity.
For testing repeatable processes and results reporting, automation certainly has its place, but it won't solve all potential testing ills for your DevOps team. There are functional tests and validations that manual testers can provide as part of feature enhancement, and deployments into different environments may not be fully covered (with an effective ROI on effort) with automation.
Automation consistently uses the same approach and the same series of steps and data to execute the same workflow over and over. Manual testers, adopting an exploratory testing mindset, have much greater flexibility to choose any means to achieve their goal. The best scenario offering the greatest test coverage will combine the two approaches.
How exploratory testing fits in
By automating most common workflows supported by traditional scripted testing, you can free up your manual testers to dive into exploratory testing, where they can quickly identify important defects and help shape the software to meet the real-world needs of end users. Uncovering major real-world defects faster is one of the strengths of exploratory testing, and it dovetails nicely with the DevOps mindset.
You need to take a big-picture view. Look at the features setup and the stability of the environments and deployments, and then come up with a good way to identify your key repeatable processes. Those are your candidates for automation. The rest of the test requirements should be dealt with using context-driven testing, with good charters defining area of focus, manual exploratory test sessions, and efficient test data management techniques for automation and manual testing.
Use mind maps to visualize the workflow and deployment strategy. Testers can identify and prioritize the key minimum viable functions and features required for the product to be considered working and acceptable by the user. Use that as a mechanism to identify and prioritize your automation candidates. You can build exploratory testing scenarios on top to make sure the software is meeting user expectations and functioning as expected from a business and workflow perspective.
A match made in heaven
Many people argue that manual testing requires more time and resources, but session-based testers are limited not by how much they need to cover but by how much time they have available. If they have X hours to conduct testing, your challenge is to come up with a set of test cases, both manual and automated, that can be run within the available time.
Prioritize your automation suite in such a fashion that you have a minimal execution window for everything, and the manual team should conduct as many exploratory tests as they can. The two run in parallel, and there's no need for exploratory testing to slow things down.
As the size of your test set grows, you'll need to constantly revise the suite, just as you would with a traditional development approach. Obsolete areas in your exploratory testing, automation, deployment, and monitoring require ongoing revision and refreshing.
With the right preparation, this blend of automation and exploratory testing will provide comprehensive coverage and leverage the best of both approaches.
Keep learning
Take a deep dive into the state of quality with TechBeacon's Guide. Plus: Download the free World Quality Report 2022-23.
Put performance engineering into practice with these top 10 performance engineering techniques that work.
Find to tools you need with TechBeacon's Buyer's Guide for Selecting Software Test Automation Tools.
Discover best practices for reducing software defects with TechBeacon's Guide.
- Take your testing career to the next level. TechBeacon's Careers Topic Center provides expert advice to prepare you for your next move.
