Teradata Terrorizes the Data Warehouse

Teradata presented several areas where they are pushing the limits, such as multi-temperature, temporal extensions, and the like. Todd Walter, CTO, did the honors. He started with hardware stuff, because hardware will change the basic assumptions of BI professionals about what they can do. As data ages, its access frequency declines...in general. There are exceptions with seasonal analysis, fraud cases, etc.

Todd shows real data on data access frequency, as shown in the figure. 43% of IO processing concentrated on only 1% of the data (as measured by storage allocation units).

Several hardware developments will impact DW architectures, such as 2.5" SFF disks, in-drive encryption, 6 Gb/s SAS drives, and enterprise Solid State Drives (SSD).

As shown to the left, there is a giant discontinuity in processor performance versus disk performance. Multi-core CPU are turbo-charging, while disk drives lag. SSD will provide performance that will rebalanced this performance gap.

An interesting aspect is that DW architects have had to buy excess storage to maintain IO performance so that part of each spindle has to be left unused if all the data is hot. NOTE: Todd checked the ratio of SSD-HDD and said, "Believe it or not, it is not a math error. HDDs are at the high end of 10**2 and SSDs are at the low end of 10**5 and the diff is roughly 150x." So, my correction to this figure is wrong. Please disregard! :)

An analogy is like a delivery truck. If you fill it up, the truck goes at only half speed. So, you only fill it half way. However, the implication is that very light stuff can go FREE in back of the truck. As a result, Teradata is moving toward multi-temperature storage hierarchies where the layers are seamless. This is an old story but with some new twists.

In the second half, Todd covered the new enhancements to the new version (due out in September) in compression, geospatial, and temporal. Fascinating discussion. I tried to listen more and type less. Teradata has previously focused on compression having zero processing overhead. That is now changing. Now it is a good tradeoff to expend CPU resources to enhance compression.

Geospatial extensions has lots of potential new applications. A discussion of the privacy issue emerged focusing on applications that locate and track customers.

Todd flew through many temporal examples, showing how the SQL syntax is greatly simplified with these temporal extensions.

My Take...

Once the new storage developments settle, the DBMS landscape will change dramatically! For many vendors, traditional DB data access layer will be obsoleted. Teradata has been talking about multi-temperature issues for years. However, this is the first time I see a strategic (and disruptive) vision. Shawn Rogers put is aptly, "This is an asteroid...a dinosaur killer!"

We should have had geospatial and temporal extensions to DW ten years ago. Todd agreed, "It is way overdue!" A concern of mine is that there are no temporal standards, unlike geospatial. However, Todd remarked that they have followed a comprehensive proposal that builds on prior research (Snodgrass et al). Unfortunately the proposal is inactive in the standards committees at this time.