September 2010 Archives

On Friday, IBM’s Bill O’Connell came to the BBBT.  His updates on the IBM Data Warehouse strategy were detailed and impressive.  Richard Hackathorn, one of my fellow BBBTers, did an excellent job of reviewing the overall IBM strategy and messaging with his blog posting.

Big Data Strategy

Looking at what IBM has announced and planned ( …unfortunately we couldn’t share/blog/tweet everything… ); IBM has a strategy that is directly aimed at being able to handle the largest scale data challenges and having those capabilities fuel solutions in other areas.  Much like auto makers use their experiences in competitive racing series like F1 and NASCAR to bring innovation and tested technologies to their production lineups; IBM appears dedicated to aligning their wide range of products to solving these “big data” issues well and seeing how those innovations enable “smaller” data industries/problems.

Yet, IBM is not looking at their innovations simply from a product perspective, but rather a whole solution view.  It is not enough for them to just bring the tools, but they are looking to make the solutions easier for their customers with best practice guidance over and above “simple” consulting.

NOTE – Merv Adrian, another fellow BBBTer, has an excellent look at the explosion of big data requirements for DBMS with Richard Winter in IBM’s Data Management Magazine.

Telecommunications Take

IBM’s strategy, both current and future, looks to solve some of the core issues for telecommunications organizations.  The twin challenges of big ( …and ever growing… ) data requirements and time to implementation pressures pose issues both to telecom IT departments and their business stakeholder counterparts.

For telecom IT departments, IBM Data Warehousing should provide the ability to ingest and process the huge amounts of data that today’s Tier 1 telecoms have available from their operations.  And as the continued move from transmission-based products to IP-based products continues to grow, this requirement will only continue to get bigger.

For telecom business execs and management, IBM is bringing the ability to analyze these huge data sets.  Described as “pushing rope uphill”, bringing multi-faceted analytics from these big data sets could be the difference between leading with telecom analytical practices or following those who can make the timely and economical implementation of those analytics possible.

IBM discussed their directions in Data Warehouse and Analytics with Bill O'Connell, Distinguished Engineer and the CTO for Warehousing Solutions. This is an update from his BBBT visit on October 9, 2009.

Bill started with a broad overview of IBM's efforts in DW and Analytics, which can be overwhelming with all the technologies, companies, products, and programs involved, along with periodic name changes. We had our usual discussion about defining 'analytics' that surfaced the usual issues, without much resolution. Bill showed the evolution of DW, as shown to the right.

The IBM Smart Analytics System (ISAS) is a packaging of analytics software, data warehouse, and hardware platform. Bill discuss the MPP architecture behind ISAS, as shown on the left. ISAS versions can be powered by System x, Power System, or System z. Interesting point was that there is so much computation power that the bottleneck is I/O speeds, even with random-access disk!

Quickly went through MPP architectures, hardware design, query optimization, partitioning, compression, scan performance, workload management, database monitoring, multi-temperature data, Oracle function support, ISAS 7700 hardware configuration, solid-state disk advantages, and then...

IBM InfoSphere Warehouse powered by DB2 was described in detail, as shown at the right. The Design Studio has many different modules, such as Data Architect, SQL Warehouse Tool, Multi-Dimensional Cubing Services, and Data Mining. Bill highlighted the support for data preparation for a Data Mining analysis as part of the Design Studio to define inputs, transformations, SQL code, and data flows. He also showed an example of integrating forecast by SPSS and wrapped up with analysis of unstructured data.

Bill concluded with IBM approach to handling Big Data for data intensive analytics. IBM took the open-source of Hadoop and built JAQL. This result in IBM's InfoSphere Big Insights. The focus is analytics on enormous volumes (much more than can be handled by today's database) of data at rest. In contrast, InfoSphere Streams does analytics on data in motion.

Today was an intensive overview of IBM's work in data warehousing and business analytics. There is so much, and it is continuously changing and evolving.