We find ourselves today in a war of worlds created as a byproduct of our success in the computing industry. We have innovated and given customers a glimpse of what is possible with modern technology. In doing so, we created a beast (read as customer). This beast wants faster innovation, constant changes and improvements, old is the new broken. If a customer app doesn’t improve (read as change) every month, it’s outdated. “Where are my new features?,” screams the beast. Though, while innovation and constant change is a must to Feed the Beast, a war of the worlds is not inevitable.

Businesses responded with a new world (read as department) of application developers. This race of people embraced change as their lifeblood so responded with new processes they called agile, continuous integration, and continuous delivery. The new world embraced source code management products like ISPW and GitHub, as well as deployment software like Jenkins and XL Deploy. These processes and products made development faster while maintaining code quality and made it possible for the new world to feed the beast and keep him happy.

UNTIL, they encountered something strange when they tried to make changes to the mainframe. On their journey there, they encountered a chasm keeping them at a distance. On the other side of the chasm was an old world of strange people known as Systems Programmers and DBAs. This strange race of people has been happily doing their job of maintaining system and ensuring database reliability, stability, performance and availability for decades without being bothered by having to directly deal with customers. This race of people is strange because they seem perfectly happy NOT to be changing anything. In fact, they seem to actively RESIST change – as long as nothing changes, the system and the database stay reliable, stable, fast and available.
Thus began the war of the worlds.

The battlefront of this war is virtually every mainframe environment in the world and the loser right now is the business, not IT. Instead of having a happy collaborative culture, the war has created an us vs. them mentality. Instead of working together, departments are not even speaking the same language when it comes to development and deployment. Communication, if it exists at all, is strained. What suffers is application deployment speed and reliability, employee satisfaction, cost control, and ultimately, customer satisfaction and loyalty.

The good news is that the battle is almost over. The new agile world is winning the battle. In the 13th annual State of Agile Report, a staggering 97% of respondents¹ said their organizations practice agile development processes. More glaring is that 33% of those had adopted agile only in the past 2 years. The scales have tipped and the last obstacle to full agile development (read the mainframe) has to adapt or fall.

When the smoke finally clears what will be left of the mainframe? Well, with 90% of the largest airlines and 87% of all credit card transactions² running on the mainframe, it is still perhaps the most vital aspect of IT Operations at major corporations. Therefore, it needs to evolve and embrace change while holding on tightly to what has made it so vital to every industry for more than 5 decades. It has to embrace agility while maintaining system and database reliability, stability, performance and availability.

By now we have all heard the term DevOps. DevOps is the area surrounding and bridging that chasm between application development and IT Operations. Proper DevOps gives a company a cohesive end to end strategy or pipeline (for those familiar with Jenkins) from planning to coding to deployment. In the State of Agile survey, 73% of respondents said they are planning a DevOps initiative in the next 12 months. To be successful, all 97% who are implementing agile should address DevOps. When a company successfully manages the DevOps process from end to end, they are successful in not just implementing a process, but in changing the culture. Worlds (departments) who were previously at war with one another are now collaborators all on the same side of the battle, working in harmony from end to end, all with the same goal of deploying quality applications quickly to continue to feed the beast!

After witnessing this war in too many mainframe environments, we do have some lessons to share to help others avoid or end the war peacefully and successfully integrate the mainframe into the agile process.

1. To be successful, the entire organization needs a change of mindset, not just a change of processes.
2. EVERYONE needs training. While mainframe IT needs training on Agile, application developers may need training on relational database best practices etc.
3. Embrace new mainframe DevOps offerings which can automatically integrate the mainframe into the agile processes.
4. Work as a unified team where EVERYONE shares in the responsibility of moving new changes to the customer.

A well implemented strategy will cohesively bring together very different groups and unify the IT organization from application development to database maintenance in what will be a very powerful competitive advantage.

To see how BMC is supporting mainframe database management with agile DevOps, please visit the AMI DevOps for Db2 page at bmc.com.

¹ The annual State of Agile Report is compiled through a survey by CollabNet VersionOne and surveys 1319 respondents from around the world and in all industries. ↩

² https://www.ibm.com/it-infrastructure/z/why-mainframe ↩

We all know that IBM Db2 has been around for a long time and was originally designed to house structured data, data that fits neatly in individual fields (e.g. name, address, phone number). In Db2 V6, in response to object-oriented databases, IBM then introduced LOBs (Large OBjects) as containers to hold unstructured data.

For more than a decade, LOBs had been sporadically used throughout the industry as needed. With v10 however, LOBs are part of the catalog and available for all. It’s a good thing too because digital transformation is driving explosive growth of unstructured data. According to IDC, unstructured data now makes up 90 percent of digital data!

Unfortunately, the implementation of Db2 LOBs is very complex with pointers between base tablespaces and LOBs, Indexes on LOBs, pointers between LOB indexes and LOB tables, and one LOB per LOB Column per partition! That means just a couple of LOBs could result in thousands of datasets and even more pointers to be concerned with in your unstructured data management process.

So what can go wrong? If normal Db2 indexes can be inconsistent with their associated Table (and hence the requirement for CHECK INDEX and REBUILD INDEX), the issues are multiplied for LOBs. There are a lot of pointers in LOBs and to prevent a disaster in your environment these pointers should be checked regularly.

Db2 LOB issues can be categorized in four groups:

1. The ROWID-Version number in the Base Table row may not be found in the LOB IX.
2. There may be entries in the LOB IX that are not referenced by any row in the Base Table.
3. The LOB data itself may not be where the LOB IX points.
4. There may be LOBs in the LOB TS that are not referenced by the LOB IX.

These groups ignore any structural problems in the LOB TS itself. Those show up when Db2 tries to process the LOB data. The most common abend is 00C90101.

To mitigate all these risks, there are three utilities that should be run regularly: CHECK INDEX, CHECK DATA, and CHECK LOB. Unfortunately DBAs rarely, if ever, run these checks because of lack of resources and complexity. The common practice in Db2 data management is to wait for the phone to ring notifying you of a problem, and then run checks and diagnostics. With LOBs, that practice is a major risk to your business because LOBs are not regularly accessed. They work more like a repository for data that is rarely accessed. By the time an application finds a problem, your copies will be rolled off and your data will be unrecoverable! Often, LOBs house data held for legal or compliance reasons. Failure to regularly validate LOBs can create a real liability for your company.

According to IBM, “LOBs are different animals” so you have pay special attention to them. Taming these animals (i.e. this data) doesn’t have to be hard, you just have to have the right technology to do it!

Whether DBA’s know it or not, their companies are going digital. Take a look at your website and see if your business is promising customers “anytime access” or “mobile access” to the data you manage. If it is, whether you like it or not, you’re now part of the digital economy!

The new digital economy revolves around the speed and availability of customer data. Data is accessed from all around the globe at all times and data access is no longer a predictable occurrence. Your data must be online at all times, or you will lose customers. With a sense of entitlement they’ve grown accustomed to, customers want instant access to their data. There are very few barriers to switching providers, and your competitors are actively working to eliminate those. With a few swipes, or clicks, or (yes) even an old school phone call, we can switch phone providers, open a new bank account, or save money on our insurance.

What does this mean to you?

According to IBM, 91 percent of customer-facing applications access a mainframe. IT executives and administrators must ask themselves whether Db2 and their most important asset — their data — are ready for a digital transformation. Db2 was once a beautiful, simple relational database. Db2 on z Systems® only vaguely resembles what it looked like in the early 1980s, though data management techniques and solutions work exactly the same way. The data we manage today is complicated by XML, LOBs, and other unstructured data. Db2 is now a high-transaction repository for enterprise-critical data, both structured and unstructured, and is capable of handling hundreds of terabytes or more of data in tens of thousands or more objects with 2.5B transactions a day. The old techniques simply can’t keep up.

Db2 customers must ask themselves if data management solutions that were “good enough” yesterday are truly good enough today. Performance gains achieved in current tools are measured incrementally, while the data and complexity of managing data is growing exponentially. DBAs are forced to create innovative work-arounds and sacrifice the needs of the business in order to mitigate the limitations of their database tools. At the same time, the mainframe space is struggling to recruit new talent. The dwindling skillset in mainframe and Db2 is an underestimated risk to corporations. The complexity involved in today’s digital data management requires deep knowledge and expertise coupled with innovative, customer-centric solutions.

Change is constant, and so are opportunities

While not always easy, change will happen and is happening around us every day. Organizations often find themselves in a reactive mode, handling issues as they arise and “fighting fires” rather than planning ahead. In today’s digital, connected world, this is no longer good enough. Customers have no patience. Database errors and performance problems will be broadcast on social media before DBAs are even aware of the problem.

And so the opportunity is presenting itself. Can DBAs be empowered to manage all this digital data? Can your company provide continuous uptime as well as the access speed expected of your customers? Are there digital solutions? The answer to all these questions is “yes.” Automation and optimization are the keys to succeed in the digital transformation. Implementing the right software tools can simplify digital data management and automate tasks for scalable and streamlined operation. With a constantly optimized database with digital tools in place, you achieve a greater level of visibility and control within Db2 and the mainframe so you’re prepared for the increase in customers seeking to access data. And working with a trusted partner can ensure that you optimize your costs as well as your infrastructure and data.

Modern Db2 environments require digital-ready data management solutions that modernize and simplify processes in order to successfully handle the complexities and demands of the digital age. Planning ahead on your part will ensure a successful experience for your customers and those who embrace the opportunity will enable unprecedented growth for their business.

Learn about Next Gen Technology for Db2 Digital Data Management

Modernize your Db2 data management to deliver new applications and services. This webinar will demonstrate groundbreaking new features for managing Db2 performance, availability, data integrity, and application responsiveness.