Big data is all the rage these days. But Insead Professor Joerg Niessing says the term is misleading. And that's only the first of many myths that can lead companies astray, as they consider the vast storehouses of data that technology can amass and help sort.
One of the shining stories of the big data era is of the supermarket chain that found men who were coming in to pick up diapers for their babies were also buying beer for themselves. So the diapers and beer were placed side by side, increasing sales. That anecdote seems to be proclaiming that if you just gather data, it will spew out glorious revenue-enhancing insights.
Not so, Prof. Niessing says. You must take control, starting with developing a strategic outlook in which you will determine how to use the data at your disposal effectively. "That's where a lot of companies struggle. They do not have a strategic approach. They don't understand what they want to learn and get lost in the data," he said in an interview. So before rushing into data mining, step back and figure out which customer segments and what aspects of their behaviour you most want to learn about.
In a recent blog post with James Walker on the international business university's website, he listed eight common big data myths but now regrets not starting with that overarching strategic issue, the largest problem to overcome. The other myths are:
Myth: Big data is big
He shudders at the term "Big Data" and avoids using the word big, preferring diverse. It's actually a large volume of data points, updated at high frequency in real-time, from various sources – or granular information, as he repeatedly calls it. The diversity of the data drives the complexity of analyzing it, as you contemplate such items as cash register information and Facebook comments. "Big Data is actually lots and lots of very small data. It's not a landslide of data; it's a sandstorm. And sandstorms can blind and disorient you," they wrote.
Myth: Big data needs to be applied right away
It's tempting to move quickly, particularly since the data arrives so rapidly, but he recommends small steps, as you develop some hypotheses and test those out.
Myth: The more granular the data, the better
An online brokerage was dazzled by all the information it had about when people traded and what they were buying. "That was too granular. You don't need to know when they trade. You need to know what they want from you when they are executing a trade," he said in the interview. All that data can confuse. Pull back for clarity.
Myth: Big data is good data
Lots of data is of poor quality, such as badly tagged photographs. Your data covering transactions are usually quite accurate but don't assume the rest are. One of your first steps is to determine which data to use – not just how it relates to objectives but how accurate it is likely to be.
Myth: Analytics are all-important
In some cases, the velocity of the data is so quick there is no time to brief the analytics team – you need fast tools that react to the information, so the cashier at a checkout can recommend another product to the customer. He also worries that people on an analytics teams may be smart but not good marketers, so be wary of the power you give.
Myth: Big data gives you concrete answers
Actually, the data is probably providing ambiguity. "The more data you have, the more likely you are to have contradictions and ambiguities that require resolution. Big Data is not all-powerful. Quite the opposite, in fact. More data gives you more witnesses, but doesn't get you closer to the truth until you leverage experienced human judgment to reconcile conflicting evidence. The future of analytics is all about combining, weighing and judging multiple sources of information and different analyses," they wrote.
Myth: Big data is a Magic 8-Ball
In fiction, the fortune-teller's Magic 8-Ball may have all the answers. And the hype over big data may have given some people the impression it's equally prescient. But first you need to have the right questions and the right data, and transcend the ambiguity barrier. He's glad the hype over big data is dissipating but worries that managers are expecting more than can be delivered.
Myth: Big data can deliver self-learning algorithms
You can automate responses to big data, as Amazon has done to recommend other products. But the number of times it has offered you foolish recommendations shows what is being learned from data needs to be refined, with human intervention and updating of the algorithms.
So celebrate what the diverse data era offers. But move deliberately, with respect for the many misleading notions out there.
Harvey Schachter is a Battersea, Ont.-based writer specializing in management issues. He writes Monday Morning Manager and management book reviews for the print edition of Report on Business and an online work-life column, Balance. E-mail Harvey Schachter
Harvey Schachter