Oscar Wilde famously said, “You can never be overdressed or overeducated.” I’m not sure about the former but I tend to agree with the latter, especially in our industry. Given the systems, platforms, and clients that we support, the learning journey is complex and continuous. And it has to start somewhere.

In 2020, I wrote that a post-coronavirus world would present new opportunities to develop the next generation of IT professionals, and new ways for organizations to access and benefit from their skills. With a move from predominantly onsite working to remote and home working, plus the opening up of a potentially bigger global talent pool, the demand for skills has never been greater—and at a time when many organizations continue to face serious skills shortages.

If current trends continue, it is likely that within the next five years, the mainframe space will have tens of thousands of unfilled jobs globally. Yet, as the BMC 2021 Mainframe Survey reported, more than 90 percent of organizations see the platform as one for long-term growth and new workloads, while 86 percent of extra-large mainframe shops expect MIPS (millions of instructions per second) to grow.

How can we meet that demand, that appetite to innovate and integrate, not only this year but in the decade ahead?

We saw this situation coming, to a degree, which is why BMC Mainframe Services established the Mainframer In Training (MIT) program a few years ago. The program has three main beneficiaries:

• First, BMC, in developing our in-house skills and knowledge base and weaving BMC products and solutions into the training.
• Second, and more importantly, our clients. The program provides them with the talent they need through a pipeline of next-generation IT professionals with on-the-job experience.
• And third, the trainees themselves. The MIT program is the start of a long and rewarding journey, taking them forward while they continue developing their skills and specialisations.

During the pandemic disruptions, we recruited nine new trainees and the training moved mostly online. We’re now pressing ahead with our plans to grow the program and take it even further to cover all the areas and solutions offered by BMC.

We have also refined and streamlined the program, which typically lasts up to three years. By targeting the issues and technologies that matter most, we compressed the first year into a high-intensity, three-month bootcamp, including hands-on work. Age demographics are working against us, and we recognized that we needed to speed up the process. Seeing-and-doing in real life is the most effective teacher—logging on and starting to understand how this world fits together, the applications, databases, subsystems, storage, and networks, and performing basic tasks to help keep things running.

Of course, two to three years is still merely an introduction as this field is so vast and complex. Every day is a school day for a mainframer, even if they’ve been in the job for 30 years. MIT is about laying the best foundations to not only underpin an individual’s career but also provide the industry with that strong base and greater confidence that the skills they need do exist.

Another important aspect is mentoring for developmental and career guidance by pairing trainees with a technical buddy from about six months in. The buddy works with their trainee on a day-to-day basis, and the relationship starts with them shadowing each other. They keep in regular contact, with the buddy dropping in to check the trainee’s work. This is where the real learning journey begins, in the real world with an experienced professional by your side.

As you’d expect, we’re also keen to promote diversity and inclusion in the program, and we do have a diverse group. Of the new cohort—four in India and four in the UK—there’s a 50/50 gender split. We want to attract all talents and recognize the value that different voices and backgrounds bring.

Representation across the board is important to me as a manager, and important to BMC. Even the words used in an advert calling for applications can impact who responds and why. When recruiting trainees, I’ve found that their attitude and aptitude are far more important than their previous experience in technology. Are they go-getters? Are they prepared to learn, to expand their minds? Will they stick with it, through thick and thin?

As I wrote back in 2020, “Our clients trust us to provide the trained people and expert services they need, when they need them. Our trainees trust us to give them the best experience we can, imparting the knowledge and skills they need to progress, to support their current and future employers. And our own people and skills base have to be more flexible than ever, ready to pivot to deliver whatever is needed.”

As this year’s BMC Mainframe Survey made clear, unprecedented business disruption and significant growth are driving digital transformations across many organizations. BMC’s MIT program is an increasingly vital element in enabling those transformations, shaping the enterprise IT and mainframe teams of tomorrow.

The headlines keep on coming. British Airways data breach: thousands of customers given more time to claim compensation. Amazon data breach fears, European e-ticketing platform Ticketcounter extorted in data breach, Oxfam Australia supporters embroiled in new data breach. And on and on it goes.

While these breaches don’t specifically relate to mainframes, it’s clear we are living in dangerous times. Especially when the mainframe continues to be the processing and transactional heart of so many organizations and is increasingly viewed as a hub for innovation. In every location where our team carries out a security audit or penetration test, we always expose significant, previously unknown security concerns. So, why is the mainframe at risk to this degree?

The people who truly “know the mainframe” and understand it have traditionally been a small group. This complexity has led most everyone else to conclude that it’s virtually secure by default. It’s not. There’s also a general lack of understanding around the detail of mainframe security by the very people tasked with keeping it secure. This leads to vulnerabilities.

As an example, individuals may not properly understand the risks involved in giving someone a superuser privilege. We have workplaces where employees are given read access to everything. But in the mainframe world, if you can read something—especially data—you can copy it. If you can copy, you can download. And if you can download, you can potentially exfiltrate the data.

In addition, the average person can’t just buy a mainframe, install the software, and start testing it. The technology is still too costly and too tightly controlled to be reverse engineered. But that’s changing as more information is shared online. Knowledge about the platform and how it can be hacked is becoming more widespread.

With that backdrop in mind, I want to share the top threats we’ve identified in the course of our work.

1. Too many users with escalated privileges. In this situation, the Superuser privilege can be inappropriately used, granting users excessive access to system services and OMVS/USS (Unix System Services) resources and data. This means data can be easily copied, deleted, or held to ransom, and the ramifications can be huge. With read access so often the norm, instead, the default access should actually be none. This is about applying the principle of least privilege (PoLP).

2. Privilege escalation vulnerabilities. Many enterprises grant excessive access to libraries and authorized datasets. This increases the risk of someone accessing your files to elevate their own permissions on the system. That could mean taking yourself from problem state, where normal user/applications run, to supervisor state: a supposedly “protected” and authorized elevated state in which a user has free rein to do all the clever stuff—or to make mischief. In the non-mainframe world, this would be called getting route-level privileges. If bad actors can get to that state, they gain the ability to read and write all data, including memory.

3. Default passwords and weak password management. Password insecurity is rife; it’s been estimated that it would take a hacker less than a second to crack eight of the ten most commonly used passwords. Password vaults are not commonly used. Organizations should not solely rely on passwords and must ensure strong password controls, avoiding static passwords and ensuring they are changed regularly. For mainframe privileged users, multi-factor authentication (MFA) is a must.

4. Access to sensitive and cryptographic data. Additional processes, procedures, and rigor are urgently required around protecting cryptographic data and keys. Read access to the database allows it to be copied and downloaded. Data set profiles that are poorly configured allow read, update, and control access. This means data can be copied, updated, or downloaded. Once downloaded, offline password-cracking tools can reveal passwords in the database.

5. “Faceless” accounts. This is another possible attack vector for hackers that occurs when the organization needs an account for a system task but there’s no real person or actual user associated with it. These accounts often come with system-level privileges. They typically have a password that is rarely changed and, if they do have a password, it’s usually easy to guess. Have you protected all of your “faceless” accounts properly and are they appropriately defined?

In most cases, weak controls and inadequate security measures are exacerbated by a number of factors. These can include insufficient headcount; inadequate resourcing or a lack of in-house skills combined with poor system configuration; and processes that are no longer fit-for-purpose or simply not in place. Many sites are running outdated (or have a complete absence of) appropriate security tools and technologies.

The best way to start securing your mainframe is to work towards a Zero Trust culture. And when you’re tackling a specialist area like this, you may need additional firepower. Few people would attempt to rewire their house or apartment and deal with the dangers of electricity without having the right tools on hand—or, indeed, without being an experienced and accredited electrician. Instead, we pick up the phone or head online to find someone who knows what they’re doing.

With so much at stake, it may be time to call in the experts.

The mainframe isn’t quite the center of the universe, but it’s not far off

The mainframe sits at the heart of the data center: it’s an essential hub, the focal point, with the other spokes of a business coming off it. It’s the reliable data server, the system of record. It’s the super-fast transaction server, the transactional beast of burden. It’s the security server for many, thanks to its innate securability.

I’m far from being the first person to suggest that the mainframe was the original cloud. If you implemented and used a mainframe infrastructure in the effective ways, you were gaining all the benefits of cloud computing decades before the term came into common usage in the mid-nineties (cloud imagery had, of course, been used for many years to represent IT and other networks).

If you define the cloud as the on-demand availability of IT system resources, in particular data storage and computing (processing and transactional) power, without direct active management by the end user—then we’re talking about the mainframe. What the cloud can do, the mainframe has been doing for a great deal longer, and doing it extremely well.

Scalability? No problem. While a major benefit of cloud is the ability to access resources as and when you need them, the sheer computing power of the mainframe means you can throw whatever you like at it. In the most recent BMC Mainframe Survey, more than half of respondents reported an increase in transaction volumes and 47% reported an increase in data volumes. 68% of respondents expected MIPS, the mainframe’s measure of computing performance, to grow. The platform can handle it.

Availability: again, mainframes match if not exceed cloud availability. Indeed, the reliability of the mainframe is a big selling point for the technology. Someone once wrote that you can measure mainframe uptime in decades. Today’s mainframe is also as flexible and agile as cloud platforms, in terms of the choice and freedoms that it gives you in, for example, virtualization and resource allocation.

Going further than ‘traditional’ cloud benefits, the mainframe is also, potentially, the most secure platform around, it just takes a little time and application to ensure it attains and retains that security posture. By contrast, cloud can present a number of security issues and challenges, including visibility and control of your data. Cloud providers treat security issues and risks as a shared responsibility with their cloud customer. A well-managed secure mainframe takes away a great many of those risks.

As last year’s BMC Mainframe Survey concluded, “The mainframe is front-and-center in today’s digital business environment” and is “viewed as a valuable, growing, and evolving platform by IT professionals and executives… a critical component of the modern digital enterprise and an emerging hub of DevOps innovation. Workloads are growing, while large organizations continue to host much of their data on the mainframe.”

“Executives and technical professionals have a high opinion of the mainframe, and they’re putting it to work more and more to support digital business.” So how can we help to make this happen?

Funnily enough, in an era of convergence and connectivity, cloud and mainframe are coming together: the mainframe in its role as the backbone of the data center, for data storage and processing, combined with cloud’s ability to bring additional innovation (in analytics, say) along with accessibility and economies to the party. This could mean making mainframe data available via a (private) cloud to the applications and people who can utilize and exploit those resources in the most effective ways.

We already know the mainframe can be truly transformative; an agent for change, a hub for innovation. But this won’t just happen, either, and we need to put the hours in.

In an Information Age article, Compuware’s Stuart Ashby suggested how we might “ignite a mainframe transformation with three key mindset changes”. Describing the mainframe as “the bedrock of IT innovation for over five decades, providing the reliability, scalability, performance, and security that organisations need”, he says organizations can address the challenges of change in three ways. First, leaving traditional waterfall thinking at the door and instead approaching incremental change and improvements in more organic and flexible ways. Second, “reaching hearts and minds” through measurement, identifying value and benefits via a “before and after” approach. And third, redefining what it actually means to develop: working towards building a software delivery ecosystem that redefines what it is to be a developer. For example, increasing the use of automation, with all of the opportunities that brings.

We don’t have to look far to see the value the mainframe already brings, and the impact it will continue to have. The platform is central to how many of the world’s leading brands are conducting their business: retailing giant Walmart (the world’s largest company by revenue), Nike (the world’s largest supplier of athletic shoes and apparel), Bank of America (one of the biggest banks and largest companies in the US)—the list goes on. Take a look around the world’s leading banks, utilities, retailers, health service providers, even the military, and you’ll find a mainframe in action. The transformation work continues.

It’s an ill wind that blows no good, and profits nobody. That old nautical phrase popped into my head when I heard about the SolarWinds Orion supply chain compromise. Yet this may be one case where something good results from something very bad.

I read recently that the hackers likely gained access using compromised credentials and/or a third party application that took advantage of a zero-day vulnerability. The SolarWinds’ CEO later confirmed that “suspicious activity” in his Office 365 email account allowed the bad actors to access and exploit the Orion software development environment. It’s believed the hackers first tested their ability to insert malicious code into Orion network management software as early as October 2019. The hackers apparently had access to the company’s emails for NINE months.

“So what?” you might say. Why does it matter for the rest of us, and the mainframe world in particular?

The hack was clearly very bad news for the eight US federal agencies affected, which included the FBI and the Pentagon, along with up to 18,000 other SolarWinds customers attacked with malware. Systems were monitored, data and IP harvested.

In fact, it’s been reported since that around one-third of the private sector and government victims of this “colossal hacking campaign” had no direct connection to SolarWinds at all. There was probably a realization by many senior IT and security people that they had escaped by the skin of their teeth. This could have happened to anyone; you didn’t even need to be an Orion software customer. That is a “near-hit” in my mind rather than a “near-miss”.

Is there a bright side to all this? Perhaps. The compromise may have some positive outcomes by shining an even harsher light on the complacency that still exists when it comes to security, and especially the different security standards that are applied to development/supplier systems (“not really important or at risk, so why bother?”) compared to in-house production systems (“we must protect our crown jewels”).

Such attitudes continue to hamper ‘Zero Trust’ approaches, and at a time when an increasingly connected world means rising threat levels through supply chain attacks. The bad actors don’t need to get to your production systems at all, which may be tightly protected. Instead, they can look to what may be a softer target: the poorly-protected dev systems of a supplier or anyone else in your supply chain. And you won’t even know it’s happened until it’s too late.

James Stanger of the Computing Technology Industry Association (CompTIA) hit the nail on the head when he described the problem thus: “Most organizations continue to pursue traditional measures based on a firewall-first, signature-based, trusted-partner mindset.” He describes this old-school BAU approach as ‘Cowboy IT’, which he defines as “underutilization of modern tools, over-reliance on old ones and a lack of proper monitoring.” Does that sound familiar?

Securing the supply chain has become a hot topic, and we can do better. We need to lift our gazes from the threats that are closest to us – once they have been mitigated, of course – to scan the horizon, imagine what else could happen in our extended environment, and ask searching questions of partners and suppliers. All systems have to be treated as production systems, with better monitoring, more threat intelligence, and making Zero Trust the order of the day – via access rights and applying the principle of least privilege (PoLP), through more rigorous password policies, using Extended Detection and Response (XDR) capabilities, and more.

SolarWinds seems to have got the memo. In moves to lock the stable door long after the horse has bolted, the company is planning to better secure itself. This includes upgrading to “stronger and deeper endpoint protection”, expanding its Security Operations Center, strictly enforcing multi-factor authentication (MFA), expanding the use of a privilege access manager for admin accounts, and increasing pre-procurement security reviews of all vendors. It’s a start.

How confident are you that your mainframe systems and data are as protected and secure as they should be?

I saw recently that the global zero trust security market is predicted to grow from USD19.6 billion in 2020 to USD51.6 billion by 2026[1], with the data security segment now leading the market. There’s a reason for that growth.

Zero trust is not an individual tool or a single platform. It’s a strategy, a security framework founded on the notion of “never trust, always verify” – or in simpler terms, “don’t trust anyone”. Nor is there an end point (pun intended) with the zero trust model. It’s an ongoing journey, a state of being that needs refreshing, updating, and nurturing. When you consider the enduring popularity and pervasiveness of the mainframe – its role as the system of record and mighty transactional “beast of burden” – it does seem that applying a zero trust approach in this world is long overdue.

In the latest BMC Annual Mainframe Survey, Security and Compliance were identified as the top mainframe priorities by most survey respondents. In fact, ‘Security’ over-took ‘Cost Optimization’ as the leading priority for the first time in the survey’s 15-year history. It’s very encouraging. And zero trust is the future we should be working towards, from access rights, password policies and insecure applications to overprivileged users, the threat of unencrypted communications and more.

One of the basic issues is that READ access is so often the norm. In reality, default access should be NONE. I believe it’s simple commonsense that you shouldn’t automatically trust anyone and anything, inside or outside your perimeters. The most appropriate course of action is to verify everyone and everything. For some years, I’ve been advocating the principle of least privilege (PoLP). Least privilege is simply about restricting access and permission rights for users, accounts and processes to only those resources that are absolutely necessary to carry out routine, authorized tasks. While that might be a significant change in mindset for some, the equation for me is a simple one:

Authenticating everybody + least privilege for all your data access, systems and applications =
Zero Trust Security for your mainframe shop.

Organizations should also be looking to improve their threat detection and response capabilities. Ask yourself the question, is an approach that utilizes endpoint detection and response (EDR) and managed detection and response (MDR) still enough in the “new normal” landscape of mass home and remote working? As we all know, the coronavirus pandemic and shift to home working has exposed the vulnerability of companies, individuals and nations to rising levels of cybercrime.

Extended Detection and Response (XDR) cyber security technology is coming to the fore. In a zero trust world, XDR might as well mean “anywhere, everyone and everything detection response”. The point is that every system, every user, every drift “from the normal” in behavior counts. The actionable threat intelligence provided through XDR capabilities could mean the difference between assured security levels or damaging hacks and data breaches.

While security should never sleep, the reality is that there simply are not enough talented and skilled mainframe security experts on the planet to constantly monitor all of our systems all of the time. That’s why harnessing automation, AI and machine learning through XDR is so important. Mainframe modernization work including AI and machine learning will mean you can pick up on anomalies and exceptions extremely quickly – because you’ve been tracking, learning from, and so better understanding previously undetected patterns.

What else? Oh yes, passwords. Don’t get me started on passwords. If you want to know more, you can have a read of my short paper on password insecurity, stolen credentials, data breaches and multi-factor authentication (MFA), The Problem With Passwords.

It’s clear we need to continue pushing back against the complacency that still exists, and the mistaken belief that the mainframe is inherently secure “out of the box”. The zero trust model is the best way to properly protect our systems and data. It’s no longer enough to slavishly follow what James Stanger of the Computing Technology Industry Association (CompTIA) describes as “traditional measures based on a firewall-first, signature-based, trusted-partner mindset.” He described that old-school approach as Cowboy IT: “underutilization of modern tools, over-reliance on old ones and a lack of proper monitoring.” I couldn’t agree more.

We need to better protect ourselves and the tools are already out there, as I’ve often said; it’s the mindset that is perhaps lacking. And it’s not as if there aren’t external experts you can call on for advice and guidance, either. We’re here to help. A a good starting point can be external pen testing to see just how secure you are, ideally followed by a more in-depth security assessment. The only thing you have to lose is… a leaky and potentially dangerous security posture. If you are completely secure and have nothing to worry about, it would be great to have that verified too. But can you be sure?

Zero trust is not only an excellent collaborative goal to build towards, founded on better analytics, automation and adaptiveness – it’s increasingly the only game in town when it comes to security.

What do you do when your car breaks down? A major home repair is needed? What if you have a toothache? Some of us might try a DIY fix, but odds are in most of these cases (especially the latter) you look to the services of an expert. Security is no different with security services now accounting for more and more of IT budgets.

The use of managed services has grown in the last few years. A big driver is that managing risk is more and more at the top of the CIO/CISO list of challenges but staffing and expertise are in short supply. Often, I find CISOs who have put Data protection, Incident response, EDR, endpoint security and infrastructure management services on their list for consideration to implement better security standards. My first follow-up question is usually “And what about your Mainframe? Who are you engaging to harden security and protect the data that lives there?” I’m usually met with a puzzled face. The mainframe has been thought of as the relatively secure box that lives somewhere in the infrastructure and isn’t tied to the corporate security initiatives. However, as a connected device with sensitive data that IS vulnerable it must be secured just like any other device.

The reality is that despite the confidence many have in the security of the mainframe, it can be compromised in as little as 6 minutes. That’s right, I said 6 minutes. And mainframes have an average of over 100 high risk vulnerabilities on them. How do I know that? I’m the guy that achieved that 6 minute mark and performed the pentests that found those vulnerabilities.

Apart from a better security posture, why else is securing the mainframe important? In the annual BMC Mainframe survey, 92% of respondents to the survey reported being audited at least every two years. 77% have been subject to a finding or potential breach. Simply put, without knowing how vulnerable your mainframe might be, you are highly likely to find out in the next couple of years and if you’re like the majority of people, it probably won’t be reassuring news. Not only that, but Forrester Research found that mainframe use is growing with 50% of respondents saying they plan to GROW their use of mainframe over the next two years. More use = more data = more risk for companies.

So what is the solution for the CISO or Security Operations person who doesn’t want to take a DIY approach to securing the mainframe? Services provided by experts, tailored to the needs of the company. Do you want to know if you’re in the 100+ vulnerability club? Consider a security assessment. Are you lacking the staff or expertise to keep the mainframe secure or just perform routine maintenance? Maybe security-as-a-service, a managed mainframe infrastructure service, ad-hoc skill services or a expertise on-demand can help address the gap?

To learn more about how mainframe services can help you better secure, maintain and address the staffing and skills challenges you’re facing, watch this webcast to learn more about where BMC Mainframe Services by RSM Partners can help!