A common thread amongst data and analytics driven organizations is to push the boundaries of discovery, outcome probabilities, and forecasts. In their quest to understand and act on every opportunity related to innovation, revenue, and savings, these visionary firms tend to rely on answers to a string of “What if” questions. And rightfully so, a McKinsey study has found that data and analytics driven firms on the average improve their top line and bottom line by almost 5%. That’s significant.
So, “What if” – a fundamental question, yet very profound in implications is actually quite difficult to answer in practice. So, what approaches do these visionary organizations use to get their answers? Many industry pundits seem to think the answer lies in “Big Data Analytics”. And they appear to be right – organizations that get their “What if” analysis right, rely on a well thought out “Big Data Analytics” strategy that captures and delivers meaning from an ever increasing volume, variety, and velocity of data that flows directly through the enterprise. In their book “Competing On Analytics”, Davenport & Harris outline several firms who have adopted a culture of analytics from large data sets.
In today’s competitive climate, it is not sufficient to understand only the past history and present status. “What if” questions are an effort to look into the future – the risks and opportunities that must be tackled moving forward. The direction an organization is headed is impacted by a large number of variables and unknowns. Understanding which variables are significant, altering the values of the variables, and plugging them into a valid model provides some potential insight. And with foresight and planning, it may be possible to alter course and be more in control of outcomes.
Let us take a deeper look at what is involved in getting to “What if” in these visionary organizations. First, they are constantly seeking out pieces of data that needs to be tagged, filtered, and analyzed to help them with “What if” analysis. To do this, they rely on Big Data originating from emails, machines (e.g. RFIDs, web logs), transactions, customer call notes, social networks, and syndicated data to name but a few. It is the job of the contemporary superheroes – the “Data Scientists” – to discover the needles in the digital haystack to find out what really matters.
Second, these “needles” or valuable data points must be put in context of the overall business issue on which the “What if” analysis is targeted, for example, stress tests for banks to simulate various edge case economic scenarios that test their solvency. It is the task of the business analysts to build mathematical models around the business issue with these valuable data points.
Third, the models need to be made consumable to the end user who will end up asking the “What if” questions under various contexts. The BI developers build these context sensitive, user friendly reports and dashboards for the end users.
Finally, the end to end “data supply chain” and workflow between these constituencies must be managed and provisioned efficiently for success. This increasingly sophisticated and mission critical orchestration is usually performed by the System and Database Administrators. So, as you can see, getting to “What if” is not trivial but at the same time it does not have to be complicated either.
While visionary organizations tend to take the giant steps in these uncharted waters, they do so with plenty of risks that can prolong or sometimes, even derail these initiatives. The key impediments to “What if” nirvana tends to be lack of a cohesive end to end plan, compartmentalized initiatives, and discrete and incompatible technology tools. Meanwhile, surveys in the recent past have found that the Big Data Analytics (and its end point in “What if” analysis), to steal a term from the youtube world, is going “viral”. So something has to change significantly for “What if” and Big Data Analysis to go main stream – something that will simplify adoption.
At Sybase, we have seen the value of data and its implications through the technology lens for more than two decades now – initially with transaction processing, followed by embedded processing, and today with analytics. Looking at the “What if” issue today, especially in the context of Big Data Analysis, we can appreciate the tools and technology challenge. We have concluded that the answer lies in providing a mature platform for Big Data Analysis that will get answers to “What if” questions quickly, consistently, and accurately.
On November 1st, 2011, we announced our latest solution targeted at “What if” and Big Data Analysis: Sybase IQ v15.4 with PlexQ® technology. The figure below provides an overview of how this new architecture comprehensively handles the multi-phase enterprise wide “What if” analysis described earlier, while efficiently dealing with the data supply chain, collaboration with privacy among user communities, and service level commitments.
I will end by saying that it is an exciting time to be asking “What if” questions. But challenges stand in the way of main stream adoption. However, at the same time, it is an exciting time to be a vendor with a vision and foresight to revolutionize the way “What if” and Big Data Analysis is done. “What if” we were to provide a lot more insights into our vision as well as practical applications in subsequent posts? This post is, after all, just the opening act.