The 2020 Continuous Testing Report: Software Testing Benchmarks

    While leaders in most organizations understand the need for, and want to adopt, continuous testing, making the transformation isn’t easy.  

    This statement is proven so clearly in the 2020 Continuous Testing Report from Sogeti and Broadcom. We interviewed over 500 senior decision makers across North America and Europe—in multiple verticals and across company sizes. 

    Some Key Findings

    The basics are still problematic:

    • 56% of organizations admitted they have challenges with in-sprint testing.
    • 65% say that requirements need clarification all or most of the time.
    • 44% of time is taken searching, managing and generating test data.
    • 36% spend more than half their time building and managing test environments.
    • 62% said they are struggling to find skilled professionals to build their continuous testing strategy.

    Year over year, teams still struggle in many of the same places, and those challenges continue to block their efforts at quickly releasing quality software.


    And, rightly, teams still have their sights set on doing more. AI to improve test cycles and accuracy was a common theme, as was the goal of increased shift-left and shift-right testing.  

    • 42% are using or plan to use predictive test selection and optimization.
    • 41% plan to do automatic defect remediation.
    • 40% want to correlate user behavior with requirements to define test strategy.


    So What Does All This Mean? 

    To move the needle, continuous testing needs to be easier: to adopt, to practice, and to evolve. 

    We need to address the basics and still do more—with an overall objective of supporting cultural transformation to embed quality into every phase of the application lifecycle. We can accomplish this by delivering easy-to-use tools that help us make continuous testing a reality—while also helping us adopt emerging technologies and advanced practices. These tools need to be flexible enough to allow teams to work the way they want to work, using technologies (like open source) and interfaces they prefer. 

    The use of artificial intelligence and machine learning holds great promise to make testing more intelligent and automated, while dramatically reducing manual effort and allowing enterprises to balance innovation and risks. We can bring cross-functional teams together, break down silos, and build collaboration around shared KPIs and actionable insights collected across systems, builds, and releases. 

    With a focus on business value, we can eliminate complexity and empower our teams to shift left, and right, and to move faster with confidence. Continuous testing doesn’t have to be hard, but it requires a culture and technology shift. As with anything else—you can’t keep doing the same things the same ways (legacy processes and tools) and expect different results (increased volume, velocity, and quality). 

    We hope you enjoy the report.