Intuition Vs Data Driven Decisions - The Evergreen Debate
Are your caught in the big debate of - Does data win over intuition for driving business decisions ?
First things first - Is intuition really independent of data points? Business leaders are generally found possess their business data on their fingertips. So, after all, their intuitive decision making is not completely independent of data. Is data driven decision making totally devoid of intuitive factors. Do the modern data science techniques uncover insights/predictions without an intuitive input. The answer is “No”!
While intuition is driven by familiar data points, data analytics is equally dependent on intuitive understanding of the data concepts and characteristics. If and when, we appreciate that each is not independent of the other, the transition to data driven decision making is much easier and more rewarding.
Some thoughts on how Intuition needs data to back it up and Data needs intuition for better results.
Intuition has popularly been called the ‘Gut feel’ and gives a notion of being judgmental! Hey, really? How many times have you experienced having lost it all by the time you explained your gut feel to others and win their confidence on the probable next move? Well, this is what we need to address. ‘Gut feel’ backed by data is more tenacious.
Intuition is only a good starting point. Let’s take an example. People dissent in your organization can be a general intuition that you carry. Can this be sufficient information for you to take the next few steps to solve the problem. You will need to know what is causing the dissent, who are the primary influencers, when did it take the turn of dissent from dissatisfaction, who are at the realm of this change. Employee surveys cannot, by themselves, give any insight, if you didn’t know in the first place what to ask your employees. Hence, intuition to gain momentum, needs understanding of the underlying data and follow the cue.
Intuition can get too compelling at times! While it is good to develop and encourage your intuitions and use them in your journey of decision making, it will be too limiting a behavior to give in to your intuitions without taking adequate steps to measure their truth. It is also true that your intuitions will gather strength when fed with relevant information to back them up. And what’s more. You start respecting your intuition as it gets closer to the reality!
“Our first impressions are generated by our experiences and our environment, which means that we can change our first impressions . . . by changing the experiences that comprise those impressions.” – Malcom Gladwell
Let’s turn the coin other way around to see how Intuition serves data too !
Data has it’s soul! Data, like people, has its soul and form. A first look at the data form can only tire your eyes until of course, they meet the data points that you are familiar with. EDA or Exploratory data analytics is a key step in the data science world and helps the data scientists to build their intuition about the data. Intuition helps us distinguish the large variety of data points from each other and their correlation to the cause or effect.
Interpretation and contextualization are still a Human concept! Do data analytical techniques fetch us direct results of “What to Do” and “What not to”. The mathematical or statistical results again bank on our intuitive abilities to correlate results to the next plausible action. Interpretation is still a big job that machines will take a long time to take over. Mathematical results will not be able to take a humanly acceptable format until their interpretation falls in place.
Well, is it not safe then, to conclude that Data and intuition have their roles in place and it would only be fair if they were allowed a free interaction with each other? However, in the world of incessantly growing big data, it is easy to tune into data reliant decisions while turning off our intuitions totally or vice versa. What we really need is a balance and a fair judgement of areas where only either of them can independently do well. In all other places, they are to be construed as “Equal share partners”.