Big Data is advertised as the secret to unlocking actionable intelligence. Collecting and sifting through vast amounts of data finds the patterns that change everything. But is elusive ‘data in combination’ the answer that we should expect from analytics? Not necessarily.
More and more often, crunching large amounts of data gets to the opposite result: The answer to many questions is found in far less data than expected. Looking at what’s being answered by large-scale analytics today, the patterns that are emerging often show surprising results like:
- A clothing retailer discovers that fit matters more than color, or vice versa
- A wine recommendation engine proves that color matters more than most other attributes, but only when a customer is an occasional wine drinker
- Only the three most recent transactions show a customer’s preferences and not their composite shopping history
Does that mean that Big Data itself is an overreaching goal for organizations? No. To understand that few factors matter, large data sets need to be created and analyzed. A Small Data answer still requires validation through data that often has velocity, volume and variety. Knowing for sure that Small Data is the answer is just as tricky, and maybe more so.
If we’re not careful, assuming complexity can blind us to the fact that simplicity is the real answer.
The key to today’s Big Data capabilities is to have an open mind and be ready for the answer that you don’t expect.