Mainframe Blog

Improving Quality and Shift-Left Testing with BMC AMI zAdviser​

Shift-Left Testing with BMC Compuware zAdviser
2 minute read
Spencer Hallman

Customer satisfaction is a top concern for all levels of an organization, from the service desk to developers, and from management to the executive suite. An organization must provide services that not only fulfill its customer needs, but that are also reliable and as free from bugs as possible. Even the smallest bug can create dissatisfaction and negatively impact a brand’s reputation.

Development teams and managers can gain insight into mainframe software quality with BMC AMI zAdviser​. Offered free to customers with current maintenance, this software-as-a- service (SaaS) solution captures data from BMC AMI DevX products and uses machine learning (ML) to develop key performance indicators (KPIs), giving users advanced metrics to help improve their organization’s mainframe DevOps processes and outcomes. These metrics can then be compared to benchmarks determined by blending anonymous data from all zAdviser customers to show how development teams are performing relative to the BMC AMI DevX ecosystem as a whole. In short, zAdviser gives development managers a view of what “good” looks like.

Two new KPIs have been introduced as part of BMC’s October release. Part of zAdviser’s Quality Dashboard, these KPIs help determine where and when abends are occurring, giving valuable insight into development quality and aiding shift-left testing.

Escaped Abend Ratio

Abends that escape to production could be customer-facing, creating dissatisfaction and damaging an organization’s reputation. The Escaped Abend Ratio KPI can be thought of as an escaped bug ratio. It shows the percentage of unique abends occurring in production compared to abends in all of an organization’s logical partitions (LPARs).

Ideally, all abends would be caught in the development testing environment, but that is never the case. This metric enables developers and management to see how many abends escaped into production and then adjust test cases accordingly to catch them in the future. The goal is to see the percentage of abends in production decrease. The same or higher number of abends in development indicates that testing is catching what it should, leading to a higher quality of product in production.

Median Time to Detect Since Compile

When a program abends, zAdviser collects a variety of data, including the amount of time since the program’s last compile. The Median Time to Detect Since Compile calculates the median (50th percentile) of this time for all abends. In a development environment, a low number is favorable—as the old adage goes, “You want to fail fast.” For production environments, the opposite is desirable; longer timeframes between compiles and abends might indicate that the program abended due to a unique confluence of events, or a unique use case that wasn’t taken into consideration during testing.

Early Warning

With these new KPIs, the Quality Dashboard becomes a sort of early warning system for developers and development management. Teams can gauge the quality of code as it is developed and deployed rather than waiting for a complete degradation of quality or a complaint from a customer. When development teams see abends increasing in production, they can refer to zAdviser’s BMC AMI DevX Total Test dashboard to see what tests were executed, when, and how they failed, then adjust test cases to make sure bugs are caught before they leak into production.

In addition to using the Quality Dashboard of zAdviser to see what “good” looks like, managers can see what “quality” looks like, as well, with new KPIs that help organizations improve the quality and coverage of shift-left automated testing and the overall quality of software. Quality products, after all, lead to happy customers.

Access the 2023 Mainframe Report

The results of the 18th annual BMC Mainframe Survey are in, and the state of the mainframe remains strong. Overall perception of the mainframe is positive, as is the outlook for future growth on the platform, with workloads growing and investment in new technologies and processes increasing.

These postings are my own and do not necessarily represent BMC's position, strategies, or opinion.

See an error or have a suggestion? Please let us know by emailing

Business, Faster than Humanly Possible

BMC works with 86% of the Forbes Global 50 and customers and partners around the world to create their future. With our history of innovation, industry-leading automation, operations, and service management solutions, combined with unmatched flexibility, we help organizations free up time and space to become an Autonomous Digital Enterprise that conquers the opportunities ahead.
Learn more about BMC ›

About the author

Spencer Hallman

Spencer Hallman is the Product Manager for Compuware's ThruPut Manager and zAdviser products. Previously, he was the Product Manager for Strobe, the Subject Matter Expert for Mainframe Performance and Field Technical Support for Strobe. His diverse experience over the years has also included programming on multiple platforms, providing technical support and working in the Operations Research field. He has a Master of Business Administration with a concentration in Operations Research from Temple University and a Bachelor of Science in Computer Science from the University of Vermont. He lives in Devon, PA and when he’s not attending virtual meetings, enjoys walking his dogs, riding his bike and spending time with his family.