Mainframe Blog

Improving Quality and Shift-Left Testing with BMC Compuware zAdviser

Shift-Left Testing with BMC Compuware zAdviser
2 minute read
Spencer Hallman

Customer satisfaction is a top concern for all levels of an organization, from the service desk to developers, and from management to the executive suite. An organization must provide services that not only fulfill its customer needs, but that are also reliable and as free from bugs as possible. Even the smallest bug can create dissatisfaction and negatively impact a brand’s reputation.

Development teams and managers can gain insight into mainframe software quality with BMC Compuware zAdviser. Offered free to customers with current maintenance, this software-as-a- service (SaaS) solution captures data from BMC Compuware products and uses machine learning (ML) to develop key performance indicators (KPIs), giving users advanced metrics to help improve their organization’s mainframe DevOps processes and outcomes. These metrics can then be compared to benchmarks determined by blending anonymous data from all zAdviser customers to show how development teams are performing relative to the BMC Compuware ecosystem as a whole. In short, zAdviser gives development managers a view of what “good” looks like.

Two new KPIs have been introduced as part of BMC’s October release. Part of zAdviser’s Quality Dashboard, these KPIs help determine where and when abends are occurring, giving valuable insight into development quality and aiding shift-left testing.

Escaped Abend Ratio

Abends that escape to production could be customer-facing, creating dissatisfaction and damaging an organization’s reputation. The Escaped Abend Ratio KPI can be thought of as an escaped bug ratio. It shows the percentage of unique abends occurring in production compared to abends in all of an organization’s logical partitions (LPARs).

Ideally, all abends would be caught in the development testing environment, but that is never the case. This metric enables developers and management to see how many abends escaped into production and then adjust test cases accordingly to catch them in the future. The goal is to see the percentage of abends in production decrease. The same or higher number of abends in development indicates that testing is catching what it should, leading to a higher quality of product in production.

Median Time to Detect Since Compile

When a program abends, zAdviser collects a variety of data, including the amount of time since the program’s last compile. The Median Time to Detect Since Compile calculates the median (50th percentile) of this time for all abends. In a development environment, a low number is favorable—as the old adage goes, “You want to fail fast.” For production environments, the opposite is desirable; longer timeframes between compiles and abends might indicate that the program abended due to a unique confluence of events, or a unique use case that wasn’t taken into consideration during testing.

Early Warning

With these new KPIs, the Quality Dashboard becomes a sort of early warning system for developers and development management. Teams can gauge the quality of code as it is developed and deployed rather than waiting for a complete degradation of quality or a complaint from a customer. When development teams see abends increasing in production, they can refer to zAdviser’s BMC Compuware Topaz for Total Test dashboard to see what tests were executed, when, and how they failed, then adjust test cases to make sure bugs are caught before they leak into production.

In addition to using the Quality Dashboard of zAdviser to see what “good” looks like, managers can see what “quality” looks like, as well, with new KPIs that help organizations improve the quality and coverage of shift-left automated testing and the overall quality of software. Quality products, after all, lead to happy customers.

Access the 2021 Mainframe Report

The results of BMC’s 16th annual mainframe survey reveals that future-forward enterprises are investing, innovating, and integrating in the mainframe, making it a fully integrated hub of digital innovation. Download the e-book for the 2021 State of the Mainframe!

These postings are my own and do not necessarily represent BMC's position, strategies, or opinion.

See an error or have a suggestion? Please let us know by emailing

BMC Bring the A-Game

From core to cloud to edge, BMC delivers the software and services that enable nearly 10,000 global customers, including 84% of the Forbes Global 100, to thrive in their ongoing evolution to an Autonomous Digital Enterprise.
Learn more about BMC ›

About the author

Spencer Hallman

Spencer Hallman is the Product Manager for Compuware's ThruPut Manager and zAdviser products. Previously, he was the Product Manager for Strobe, the Subject Matter Expert for Mainframe Performance and Field Technical Support for Strobe. His diverse experience over the years has also included programming on multiple platforms, providing technical support and working in the Operations Research field. He has a Master of Business Administration with a concentration in Operations Research from Temple University and a Bachelor of Science in Computer Science from the University of Vermont. He lives in Devon, PA and when he’s not attending virtual meetings, enjoys walking his dogs, riding his bike and spending time with his family.