IBM: Consolidate smarter with the Data Warehouse ecosystem
IBM’er Nancy Kopp talks about consolidating smarter by matching the right compute capabilities within the Data Warehouse ecosystem to the workload where it’s most optimized. Enter Nancy:
———————————————————————————————-
What’s IBM thinking in regards to a new way forward in Data Warehousing?
Sometimes you have to stop and think about where you came from to get a good direction forward. I think we learned a lot in our journey to consolidate our data marts. Heads up, the dynamic environment of our business is pushing us towards a new strategy.
Consolidate based on best practices (not just everything)
Let’s be realistic, maybe we don’t consolidate everything into one big structure. Instead, let’s apply certain principles or methodologies to what we do consolidate. Allow offloads of the Enterprise Data Warehouse (EDW) where it makes sense- especially for analytical applications that the business wants quickly.
I know this is the opposite of what Oracle and Teradata are telling you but it’s time to call it out— those big monolithic structures did not answer all our prayers in data warehousing. The overall success rates of EDW projects were incredibly low. Why? It’s really a combination of creating too much complexity and the failure to deliver new value in a timely manner.
IBM’s vision for the data warehouse architecture going forward is based on the concept of Smart Consolidation and leveraging analytical appliances within an EDW Ecosystem that is fully optimized for delivery of analytics.
What is Smart Consolidation?
We can consolidate smarter by matching the right compute capabilities within the Data Warehouse ecosystem to the workload where it’s most optimized.
This is different from consolidate everything. In our rush to consolidate, we lost the hearts and minds of business because we created a demand for the data and then couldn’t respond quickly enough.
Let me give you a real life example. We had a client who had built a fairly complex EDW system. Over time, they started to be challenged with query performance and the client was quite anxious to roll out some new applications in sales and marketing. Surprise surprise, the business did not want to wait. Rather than frustrate the business with an increased timeline, this client chose to offload to a Netezza appliance. The result? Queries ran 24 times faster and they were able to achieve lower TCO. When it comes to the EDW, we all want to do a few glory marches a year with our business users- come on admit it!
Think about this….
When you go to Google to look for information, would you accept an answer of “we’ll get back to you in 3 months?”
Ok, I didn’t think so. As IT professionals we have to focus on being responsive to our business needs. Our business users have the “google mindset” – they just aren’t going to wait for the answers they want. We need to increase our agility in data warehousing.
So consider this- the Enterprise Data Warehouse (EDW) is now part of an intelligent business ecosystem. This ecosystem offers all the capability you need to deliver business intelligence and maintain governance overall. It allows you to match the work within the ecosystem with the right computing solution AND it helps maintain performance across workloads. So whether you need to analyze unstructured data, leverage a queryable archive or increase the efficiency of some targets analytics – we can help. This is the evolution I am talking about!
You are probably wondering HOW we would do this? The answer as any good Data Warehouse Architect knows is in the overall design of data architecture. What we are talking about is a smarter way at accomplishing the data integration and delivery for the enterprise. Our brilliant counterparts on the IBM Global Services Business Analytics Organization (IBM Global Services) team have come up with a new way to tackle this issue today, with a new generation design.
This design features a “data synergy hub” that allows us to maintain lineage of the data as well as an atomic layer or single source of fact- but also take advantage of appliances where it makes sense. Of course everyone always wants that black and white answer of where does it make sense and I am going to repeat something very important. Your business requirements need to drive your design, not the technology. If you need some help being business requirements driven, call the GBS team, they can help contribute to your EDW success.
This new way of thinking in regards to both design and the infrastructure allows us to leverage and deliver more sophisticated analytics within the ecosystem around your data warehouse. Yes, you read that right- we are evolving thinking from a single data warehouse architecture to an ecosystem around the EDW that allows us to have consolidated and governed enterprise data but allow offloading to other systems that can help the goals of delivery and performance, especially for analytics.
It’s brilliant, it’s beautiful, and it’s the answer we have been waiting for.
Be a part of a “Smarter” Data Warehousing Revolution
Demands in our business today require more agility and more focus on time to value. The “old way” of creating a large monolithic structure really complicated our ability to deliver. We started to focus too much on managing performance as opposed to delivering value. So continue down the path of consolidation for the right reasons. Aim for that single source of fact but accept that the answer may be to leverage new appliances and technology in your ecosystem to accelerate delivery.
Think about how you articulate this vision to your company. New tools like the IBM InfoSphere BluePrint Director have been built for that very reason. Now the Information Architect no longer has to guard his whiteboard masterpiece from the night cleaning crew. That vision can be better articulated and tracked through the progress of the project. Get those drawings off the whiteboard and into a tool where you can track progress, requirements and design changes.
Wow things have come along way. What are you waiting for? Join us!!
Learn more about IBM’s Data Warehouse Appliances
———————————————————————————————-
See more from Nancy on her blog at NancyKoppOnDataWarehousing.wordpress.com.
Pingback: The cathedral in the bazaar: the future of the EDW — Too much information