When discussing the pros and cons of software development, especially the testing side, much of the focus is placed on the process. But sometimes it helps to see, accurately and clearly, just how much money and time is saved in the end, so  BMC commissioned Forrester to conduct a study of cost savings and business benefits enabled by BMC AMI DevX Total Test. To do this in a way that would drive home some numbers with unmistakable clarity, Forrester interviewed four BMC customers—a total of seven decision makers—and then compiled their responses into a composite profile, which it calls a Total Economic Impact™ (TEI) study.

Manual testing is both time-consuming and unreliable

Prior to adopting BMC AMI DevX Total Test, the customers surveyed had all relied on manual testing , which presented significant challenges. Because the organizations operated in highly regulated environments and their mainframe code supported business-critical systems, rigorous testing of code changes was required. In some cases, changes were still being made to programs that were decades old, with some containing millions of lines of code.

Manually performing thorough integration and regression testing on such a scale was not only time-consuming—the customers interviewed typically budgeted about three days for testing in each ten-day sprint—it was also unreliable. Given such a high volume of tests, some errors went unnoticed, and in some cases, testers accidentally introduced errors during testing. The pressure to innovate, develop, and deliver code faster exacerbated these issues.

The study also found that not only did manual testing take time and introduce errors, it also took time away from opportunities for innovation, such as enabling developers to write code that supports new business logic. In other words, it’s not just about the time spent doing an outmoded procedure that needs to be addressed, it’s also about what they’re not doing that’s costing a company in time, money, and competitiveness.

For example, from a team perspective, writing test scripts took about as much time as performing a manual test. But unlike a manual test, the test scripts were reusable and could be run automatically. This meant that when a program had to be changed, they were able to save up to 90 percent of the time that had been dedicated to manual testing. Not only was the speed and quality of code development increased, the time saved was reinvested.

The four big numbers

In terms of quantifiable benefits, Forrester estimated the risk-adjusted present value of BMC AMI DevX Total Test over three years to be:

• $4.5 million worth of time savings and increased developer output, or about one day every two weeks.
• Twenty percent fewer bugs—saving $376,000—by finding bugs that manual testing would have missed, and finding them earlier.
• $2.9 million saved with automation efficiencies that allowed more tests in less time, and improved code quality.
• $2.6 million saved by reducing the risk of major mainframe defects and preventing negative customer impact from bugs in production.

From this analysis, Forrester determined that the composite organization yielded benefits of $10.34 million on an investment of $3.4 million, for a net present value (NPV) of $6.95 million and an ROI of 205 percent.

The marketplace

The research was conducted in the banking and financial services industry and the healthcare sector, both in North America and EMEA, with companies that had developer populations in the hundreds, and in one case, numbered 2,000. It is a crucial time for businesses seeking to recover from the impacts of the global pandemic and maximize their potential for long-term growth and sustainability in a period of high-velocity digital transformation. The relevance of mainframe technology in supporting the huge demands of data and speed in every industry is beyond question, but the constant caveat has been the need to update the testing process to bring it more in line with distributed technologies.

As the survey demonstrates, the benefits of incorporating automated testing cover both sides of the marketplace—improving quality and innovation inside a company’s walls while delivering a reliable product to the end users outside.

To learn more about the benefits and savings realized by these customers after adopting automated testing with BMC AMI DevX Total Test, join our October 7 webinar and read the full Forrester TEI report. A one-page infographic summary is also available. To calculate the potential financial impact of Topaz for Total Test for your own organization, check out Forrester’s companion Business Case Builder.

The need to provide customers with outstanding experiences is obvious, but the need to provide developers with modern tools and processes may be less so. As mainframe developer demographics continue to shift and the acquisition and retention of top talent proves to be an ongoing challenge, organizations must consider the developer experience as they plan and undertake the digital transformation necessary to remain competitive in the modern digital economy.

Today’s developers are knowledge workers, looking to contribute to customer satisfaction through continuous innovation and creation of new features and services. The ability to do so in a modern environment, with tools and processes that empower them to create meaningful applications, can help attract and retain top talent in your organization.

Recent survey data, however, indicates that the mainframe developer experience is generally subpar. A new Forrester Consulting thought leadership paper commissioned by BMC, “Modernizing Mainframe Development Tools Can Help Drive Greater ROI,” shows that 79 percent of developers said their organization’s mainframe development tools need significant improvements to be more effective.

As your organization looks to extend critical business applications, it is necessary that you provide mainframe developers with modern tools, practices, and feedback. In short, you should treat your developers like high-performing athletes by encouraging creativity, providing them with the tools they need, embracing an agile mindset, and delivering meaningful feedback that helps them improve their performance.

BMC AMI DevX’s April 2021 launch helped organizations achieve these goals with an integration between BMC AMI DevX Code Pipeline and Visual Studio Code, new integration points and APIs that enable an open borders approach to mainframe development, and more. New features in our July launch build upon this commitment to helping you provide mainframe developers with the best, most modern experience possible.

Integrations with GitHub Actions and HCL Launch

A new integration enables GitHub Actions to call BMC AMI DevX Code Pipeline directly to compile, build, and deploy mainframe code. This integration leverages the power of GitHub to simplify the DevOps toolchain while making it easier to build CI/CD pipelines with automated workflows.

New capabilities with HCL Launch extend BMC AMI DevX Code Pipeline’s built-in integrations with industry-leading release management tools, further enabling users to employ their tools of choice and facilitating integration of the mainframe into their enterprise release management processes.

BMC AMI DevX Code Pipeline sandboxes

Developers can now have their own isolated build and deploy environments in BMC AMI DevX Code Pipeline, giving them the flexibility to easily make code changes in isolation, independent of changes other developers are making, then test and merge these changes back into their continuous integration system.

Improved editing functions within BMC AMI DevX Data Studio

Improved BMC AMI DevX File-AID editing functions make copying and pasting easier within BMC AMI DevX Data Studio by allowing users to copy and paste multiple cells and between editors.

Expanded automated testing coverage and security

A new palette item within BMC AMI DevX Total Test increases the coverage and capabilities of automated test scenarios by enabling the execution of REST services within test scenarios. Expanded capabilities also include load testing and 3270 automation within automated test scenarios.

We’ve also added the ability to use client certificates for authentication within DevX Total Test and from Command Line Interfaces, adding additional security capabilities into your DevOps toolchain.

New BMC AMI zAdviser dashboards

BMC AMI zAdviser helps organizations know exactly how they’re doing as they strive to improve the quality, velocity, and efficiency of their software delivery. By collecting usage data from BMC Compuware (and now BMC AMI) solutions, it enables users to measure both overall performance and the performance of individual developers against Key Performance Indicators (KPIs), providing valuable insight and meaningful feedback.

A new quality dashboard within zAdviser features new KPIs. “Escape Abend %” enables users to see the percentage of unique abends that occur in production compared to the overall number of unique abends. “Time to Detect from Compile” shows the time taken for an abend to occur from the date of the program’s last compile, while “Median Time to Detect from Compile” displays the 50^th percentile of the time to detect an abend since the compile date.

BMC AMI Ops is now the first BMC AMI product captured in zAdviser. The BMC AMI Ops Automation Dashboard displays captured zAdviser data for customers with CMSC running on the same LPAR on which BMC AMI Ops is installed.

BMC AMI Ops Automation for Batch ThruPut enhancements

ACM for Tailored Fit Pricing constrains less-important workloads on busy systems, lowering CPU busy percentage, thereby enabling TFP customers to manage CPU costs.

Scheduler Intelligence’s Late Job Acceleration detects if a job is going to miss or has missed its due out time, allowing BMC AMI Ops Automation for Batch ThruPut to increase the targeted job’s service class by one, thus giving the job more resources and helping it complete sooner.

Modern tooling for the modern mainframe

By offering a simple connected experience for the mainframe and enabling automation everywhere, these new features allow organizations to standardize and automate processes that are currently being carried out manually—and inefficiently—while making it simpler to modernize the mainframe and connect it with modern toolchains. And by offering functions and work environments that make mainframe development easier, they enable the acquisition and retention of talented developers who know that developing on the mainframe will be just like working on any other platform.

As you modernize your mainframe environment with DevOps and CI/CD pipelines, don’t overlook the importance of modern tooling in creating an experience that attracts developers and enables them to do their jobs to the best of their abilities. Your developers—and your customers—will thank you.