Industry and DBA Trends
With this blog post I am kicking off a new series of posts on the topic of how and why DB2 for z/OS is changing, and what these changes mean in terms of managing, administering, and developing DB2 databases and applications. The basic premise, which if you’ve been working with DB2 lately I’m sure you’ll agree, is that the DB2 environment of 2016 is different than it was 15, 10, or even 5 years ago. The industry is changing, the way that DBAs work is changing, and DB2 is changing – and we all need to come to grips with the fact that the way we worked with DB2 in the past is no longer the way we work with today’s modern DB2.
Let’s start our coverage by looking at some of the industry trends that are impacting IT and business and thereby contributing to the changes that IBM has been making to DB2 for z/OS.
The first, and most obvious trend, is that we are saving more data than ever before. According to a recent study by IDC the digital universe will continue growing at 40 percent a year into the next decade. By 2020, IDC estimates that the digital universe will grow to 44 zetabytes, or 44 trillion gigabytes. For clarification as to the size of a zettabyte, consult Figure 1.
|Abbreviation||Term||Size||Power of 2|
|KB||Kilobyte||1,024 bytes||2**10 bytes|
|MB||Megabyte||1,024 KB||2**20 bytes|
|GB||Gigabyte||1,024 MB||2**30 bytes|
|TB||Terabyte||1,024 GB||2**40 bytes|
|PB||Petabyte||1,024 TB||2**50 bytes|
|EB||Exabyte||1,024 PB||2**60 bytes|
|ZB||Zettabyte||1,024 EB||2**70 bytes|
|YB||Yottabyte||1,024 ZB||2**80 bytes|
Figure 1. Data Storage and Size Terminology Chart
This data growth is occurring for a variety of reasons. Organizations are rapidly moving more of their business online to take advantage of the immense popularity and growth of smart device usage (smart phone, tablets, etc.) And more “things” are being connected to the Internet (Internet of Things, or IoT), which can produce data streams that are collected and tracked. The explosion of data on social media (e.g. Twitter, Facebook, etc.) that is being tracked for marketing and other purposes is also contributing to this vast uptick in the amount of data being created and stored. And we must add all of these factors to the increase of data required due to business growth and expansion. We are truly living in the Big Data era.
The ability to collect such large amounts of data has given rise to the Big Data industry trend, which is in itself a collection of trends. The essence of the Big Data movement is being able to derive meaning quickly from vast quantities of data – both structured and unstructured – in order to improve business decision making. So Big Data encompasses, embraces and collects together the following trends and capabilities:
- Business Intelligence – structured queries and reporting
- Cloud Computing – access to large pools of computing power available as needed
- Distributed data – data is usually physically distributed across a network using inexpensive commodity hardware
- NoSQL and Hadoop – new data persistence methods geared for storing and processing large amounts of data
- Analytical tools – for data from multiple sources and of variable types
- Networked devices – the number of networked devices overtook the global population of humans in 2011
- Sensors – more sensors producing more data more frequently
- The Internet of Things – machine-generated data read and used by other machines
Big Data represents a major shift in IT requirements and technology. The phenomenal growth of data, and the speed at which it is being generated offers an opportunity for uncovering information and trends that were heretofore unknown.
As a final note here, IDC estimates that the amount of unstructured data (that is, data that is not strictly numeric, character or date/time) accounts for 90 percent of all digital information. And increasingly that unstructured data is finding its way into our database systems.
Database administration is another lens through which we can view the current state of data and DB2. The DBA, traditionally, is the technician responsible for ensuring the ongoing operational functionality and efficiency of an organization’s databases and the applications that access that data. But modern DBAs are relied upon to do far more than just stoke the fires to keep database systems performing. Most DBAs have years of IT experience and work on technologies related to database, too. This can span areas as diverse as application development, middleware implementation, transaction processing, business intelligence, and networking. So today’s DBA is not always just a DBA, and that means that the DBA is not always 100% devoted to the care and feeding of the database.
An additional trend is that there are far fewer single-DBMS DBAs than there used to be. Twenty (or even ten) years ago, it was common for a DB2 for z/OS DBA to work on only DB2 for z/OS. But increasingly in this day-and-age of heterogeneity many DBAs are tasked with managing multiple DBMSes (DB2 and Oracle; or DB2 z/OS, DB2 for LUW and IMS; or even DB2 and a NoSQL database). When focus is diluted in this manner it becomes difficult for DBAs to be the DBMS-specific experts that developers and management expect them to be.
On top of that, fewer DBAs are being asked to manage more data. As we discussed in the first section of this post, more and more data is being stored and accessed, but that is not translating into additional DBAs being hired. Although the Bureau of Labor Statistics projects the growth rate for DBAs as “faster than average” at 11% between the years of 2014 and 2024, that is still less than 2% per year. And when we match that up against the 40 percent a year growth rate for data into the next decade, it is easy to see that organizations are not planning well if they hope to keep a handle on their burgeoning data stores.
It’s Not Your Daddy’s DB2
All of these big data and DBA trends factor into the changing world of DB2 for z/OS. In subsequent posts over the next few months we will examine some of the biggest changes that organizations and DBAs are facing with DB2 for z/OS. We will look at database structures like Universal table spaces and LOBs, hybrid transaction analytical processing with IDAA, SQL development trends and even how autonomics are changing the way we manage DB2.