During a recent proof of concept of our next generation technology utilities, something struck us as very significant: One of the test cases included a billion (with a “B”) row table. Not so long ago, million row tables were the largest we saw in tests. Given that, is it unreasonable to expect trillion row tables by the end of the decade? Continuous advancements in digital business, the Internet of Things, compliance and regulations, etc. are forcing the creation of more data in a few days than has ever been generated in history!
DB2 is prepared for this volume. Think of the new types of table spaces like UTS PBG (partition by growth). A PBG can have up to 4,096 partitions. With a scale like that, 200M rows in a partition is extremely close to the 1T row number. If that sounds ridiculous, we should point out two things: We have customers now with that many rows in a PBG partition; and there is no evidence that these objects will stop growing, especially given the estimated mobile growth over the next five years. The maximum size of a PBG partition is now 256G so in many cases they can handle far more than 200M rows.
Why should you care? You’ve been hearing this for years, right? The problem is that data volumes are increasing faster than traditional methods of data management can accommodate. Sorting a compressed single partition of 256G could require 1T of DASD. Remember that traditional methods require a PBG to be treated as a single dataset, meaning a reorg of a single partition is not extremely useful.
The old methods of reorganization (unload/sort/reload) will become increasingly insufficient. Constant performance improvements offering small percentage gains will wither under the onslaught of massive increases in numbers and sizes of DB2 objects. Over the next 5-10 years, there will be an increasing number of DBAs who have objects they will not be able to process. They will be faced with the dilemma of either radically changing the way they store the data or sacrificing application performance and customer response times. Modern complex DB2 databases require a new data management technology.
21st Century DB2 data management does exist and those who are embracing new methods of data management are also reaping the benefits of reduced costs, improved app delivery times, and increased availability. Most importantly, these forward-thinkers are prepared for the reality of massive data growth on the horizon.