I’ve been talking to a lot of folks lately on Big Data. One thing I keep stressing with any talk on Big Data is that most Big Data is big for a reason: it is a high volume, low value type of data. What I mean by that is a single element (or transaction if you will) of something in the realm of Big Data is, by itself, not very interesting. It is the patterns and trends that materialize out of looking at massive amounts of these elements that start to become interesting.

However, this analysis on just the Big Data is never enough to tell a full story. These Big Data sources (RFID and sensor, social network, web page, geospatial, scientific, multimedia, and unstructured communications (email, video, chat, etc.)) are inputs to an overall analysis we need for decision making. We need to add the Big Data trend analysis with data from more traditional, low volume, high value data sources. These high value transactions (customer records, order and payment records, and others where individual elements are of significant business or organizational importance) are more ordered and structured in definition.

So – Big Data is not alone – it is part of all managed data, and needs to be understood together with highly structured elements. In other words, any effort to model and use Big Data needs to integrate with the efforts we use to architect information for the enterprise. Techniques like business glossaries and conceptual models that define business information context will be essential to aligning Big Data sources to match all other data sources. We need to unify the understanding at a high enough level of abstraction so that we can come up with a unified view. Big Data is nothing without the rest of the available information lending full context – and modeling (not physical but conceptual) will be essential to align everything.