Here’s a peek into what continuous testing means to DevOps today
The “2018 Accelerate State of DevOps” report is the brainchild of Dr. Nicole Forsgren, Gene Kim and Jez Humble at DORA (DevOps Research and Assessment). Based on five years of research, with more than 30,000 data points from thousands of companies, the project aims to understand precisely what practices enable teams to deliver better software faster.
The 2018 report places an increased focus on testing and quality, including a new report section dedicated to continuous testing. Here are three key report takeaways that are important for continuous testing:
1. Testing must change—significantly—to keep pace with DevOps
Consider this:
- Elite performers deploy code on-demand, multiple times per day.
- At elite performers, less than one hour elapses between code commit and production.
The rate of change is accelerating—fast. Testing must somehow keep pace. Evolving testing for DevOps is critical for realizing DevOps’ full potential in terms of accelerated delivery, agility and all the derivative business outcomes outlined in the previous point.
2. High rates of test automation (~90%) are essential for DevOps
At elite performers, manual testing represents only 10 percent of the total effort. Even the 30 percent manual testing rate for low performers is significantly better than the industry average of 80 percent+ manual testing (including teams that have not yet adopted DevOps).
The importance of test automation is underscored by the findings of a recent Forrester report that’s focused specifically on what quality practices separate DevOps leaders from DevOps laggards. That research found that DevOps leaders are:
- Transforming software testing into continuous testing by adopting five core practices, such as automating end-to-end functional testing and integrating testers into cross-functional teams.
- Almost twice as likely to consider automating the software quality processto be a “critical business differentiator” (53 percent vs. 27 percent).
- Significantly more likely (by 23 percentage points) to have high levels of automation for key testing and QA processes (test case design, functional test automation, test data management, etc.).
3. Quality is correlated to DevOps success
Elite performers have a 7X lower change failure rate than low performers. They also spend 2X less time fixing field-reported defects. Exactly what DevOps practices are contributing to the impressive quality outcomes being achieved in the top-performing organizations? And how are different quality practices contributing to DevOps success? This is certainly an area that’s ripe for further research and analysis.
Want to learn more about this report’s findings—and what it all means for software testing?
Join Nicole Forsgren, PhD, CEO and Chief Scientist at DORA and Wolfgang Platz, Founder and Chief Strategy Officer at Tricentis, for a webinar to explore “Continuous Testing’s Role in DevOps success.” The focus will be on helping IT leaders understand:
- Why and how high-performing teams make Continuous Testing a linchpin of their DevOps process.
- What specific strategies and practices enable high-performing teams to deliver faster and dedicate more resources to innovation.
- Which quality-related technology and process investments have the greatest impact on DevOps success.
