This season brings us a comic book entertainment twist on the big screen, with superheroes pitted against each other in life and death battles. Marketers raise our interest and compel anticipation by goading us fans into “taking sides”. The approach may be novel to movies but it’s a long-standing and proven approach to sport where intense rivalries are always central to the plot. Remove home team mania, my hero versus yours, and we will still have excellent sport but accompanied by empty stadiums.
I see a similarity unfolding in business. There are new superheroes in business called Data Scientists, trained in the crafts of Engineering, Mathematics and Computer Science. They are a different breed than found in Market Research, represented by the various disciplines of Social Science and Statistics. While fighting for a common data analytic cause to improve insight and compel better problem solving, and against the common scourge of decision making by gut instinct alone, there’s something to be learned pitting the powers and approaches of these two disciplines against one another.
Special Powers of Data Science
Let’s look first to the special powers of our Data Scientists. Underneath their nerdy, Clark Kent like exteriors, there are considerable and special powers that get unleashed to combat poor Decision-Making. They bring computational capabilities faster than a speeding bullet, strength to lift volumes of data previously unimaginable, vision that can see through both structured and unstructured data, and the ability to jump over the largest functional silos of business in a single bound.
Applications of their powers are versatile, spanning all corporate functions.
Think of Finance and Fraud Detection. Old systems run with a series of hard coded rules, set based on a particular analysis done in a point of time, and occasionally updated as the bad guys figure out how to game the system. With Data Science, you now take in all historic data available while live streaming new data as it arrives. You build and expand out far more variables while constantly improving your predictions based on probabilities. You calculate based on what people say or write, not just the numbers they represent. Then you create Machine Learning algorithms to interact real time with your business systems and/or staff such as your customer service reps, presenting real-time solutions and decisions. The result: greatly improved accuracy to pre-empt cheaters or those who will default on their commitments, while also expanding offers to customers who otherwise would have been neglected.
Think of Marketing and your Marketing Mix. Old systems relied on point-in-time segmentation models, built out the offers within the business plan contemplated, and served these up through advertising, sales or customer reps and web portals with a set of hard coded rules governing a hand full of offers for the segments identified. With data science, the ability to scan the market far more comprehensively, expand and mathematically improve segmentation models, and then determine near instantly what offer or personalized content to put in front of each customer in every channel and circumstance is now possible.
Think of Operations and your Supply Chain. Data science has allowed massive customization of the retail and grocery landscapes for multi-unit enterprises. Warehousing, orders and deliveries, product SKU mixes, prices, promotions all have demands that change daily store to store with Data Scientists directing this traffic. Think of HR and how Data Science can help you attract, onboard, train, engage and retain talent needed to succeed. The list goes on. Wherever there is a chain of events to explore and to optimize, Data Science is there with its powers to help and improve.
Special Powers of Social Scientists
Let’s take a closer look at the powers of Social Scientists, the world’s Greatest Detectives. Batman does not possess any superpowers; rather his relevant (to this exercise) skills are a reliance on intellect, detective skills, knowledge of scientific methods, an affinity to technological gadgetry, and an indomitable will to seek and to hold to a scientific truth, irrespective of the political purposes of those being served. He does not sit on the Town Council. He gets called on by leadership but is not beholden to them. Truth seeking is not a comfortable task.
The point to be taken overall is that these powers are human and largely focused on unlocking the riddles of human judgement and decision making. Rather than an obsession on what people will do, it is a discovery process of why they do what they do. Do not be misled by the movies who always underplay it; most hours of social scientific detective work are spent toiling in a lonely bat cave.
Therein lies the Social Scientist’s strength. They are detectives solving human riddles. They understand what motivates and shapes desires, good and bad. They design tests and experiment with a disciplined method that attempts to disprove their own theories to ensure rigour in real world applications. While they share an interest with Data Scientists in the observation of human behaviors (data), their real expertise is in understanding the cognitive and emotive antecedents of these observations. What are the cultures and the beliefs, the attitudes and the perceptions, the emotions and sentiments that drive human decision-making? A course in Psychometric Statistics is therefore more helpful here than in Mathematical Engineering.
Analytics Battleground
In approaching problem solving, the Data Scientists’ meta-methods are largely Inductive. They can utilize virtually unlimited number of observations from which they discern a pattern, then make a generalization, and finally infer an explanation or theory.
The Social Scientists’ approach is diametrically opposed, with a generally Deductive approach. They begin with a theory, followed by a pragmatically limited number of observations, from which they expose a pattern that either refutes or supports their theory which leads them to a specific conclusion. The battleground between Data Scientists versus Social Scientists, if we can enjoy contemplating one, isn’t one of technical know-how. Rather, their difference stands on how they approach a problem set.
It is anathema to a trained Social Scientist to go on an insight fishing trip, correlating everything with everything with partially filled data in the hopes of finding something of (p) value. They are taught that these correlations are spurious by nature and that deep understanding of what is observed is a requirement before looking at its relationship to others. In what appears as stark contrast, for Data Scientists most of the exploration is done in higher order spaces, generally the more data the better, and only a weak prior understanding of its nature required. Nonetheless, with interconnected higher order relationships the goal, much gets explored and done.
Social Scientists toil with little glamour, but there should be no schadenfreude — Gartner Analysts inform us that despite their superhero status, the business craft of Data Science has fallen into a zone of disillusionment. Some of this may be circumstantial. Excessive hype led to growth that outstripped the available talent pool and inevitably clients may have been oversold and projects may have underperformed. Some clients, I venture, may not have coupled their grand ambitions with an adequate understanding of the process of discovery. Lock an exploratory Data Science team in a room for six months with few boundaries other than to make a break-through discovery and they may return you Thomas Edison’s discovery: “I have not failed. I have just found 10,000 ways that won’t work.”
But the simple truth for some who have been disillusioned is that they may have misunderstood how and when to apply Data Science. Those committed to the craft understand that despite all the powers Superman possesses, blunt strength and speed are not enough. While Superman flies around looking for trouble, Batman waits in the cave until a potential answer to the riddle comes clear before venturing out with a testable set of hypotheses to embark on discovery.
I therefore offer the provocation that Data Scientists and Social Scientists are “frenemies”, both trying to convince powers above of the value of insights, to save us from ignorance and poor judgement, though through very different approaches.
Beating the Common Enemy: Poor Decision Making
While a battle among superheroes may have drawn us in, let us not forget the common enemy to be fought — poor decision-making. Batman — the World’s Greatest Detective — working alongside Superman can win in ways previously unimaginable. Return to our Finance use case on Fraud detection and let’s involve the market researchers. What sort of circumstances invite lapses in moral judgement that result in fraud? Which of these are situational and can be discouraged versus those that reflect character flaws of prospective customers that must be avoided, and how do you tell the difference? How have all the stakeholders involved been aligned with compensation, recognition and rewards, to encourage the right behaviours? What kind of brand have you built that encourages and attracts the behaviours and people you want to see as your customers and your employees? Both disciplines are well versed in Predictive and Prescriptive Analytics. One should be more opinionated about the user experience and the psychological antecedents of fraud; the other poised to turn bullet points on a Power Point chart into an algorithm built into software to actualize the learning in real time.
When contemplating your Marketing Mix, and prior to debating methods of clustering and distance metrics in the application of statistical techniques, bring along your Social Scientists. They will be well versed on the basics and nuances of segmentation types (e.g., a priori, usage, attitudinal or need). While the Data Scientist will lend further statistical heft and computational ability to test, experiment and deploy, the Social Scientist ought to have considerable opinion on content and approach that would otherwise be missed. These powers need to be brought together to work in tandem and should not be segregated by talent. The same rationale applies to other corporate functions such as Operations and HR. I believe the point is made.
Data Science is most effective when it is used in combination with the unrelenting detective work of Social Sciences. Blunt computational power and statistical strength serves us well when we already have an understanding of the right questions to ask. A choice of database platforms and statistical applications should be secondary decisions. Form should follow function. Cross disciplinary thinking across silos is needed to win at evidence-based decision-making. We pretty much know how this season’s superhero movie will play out. Our superheroes will have to work together to save the day. When it comes to decision-making, we need to do the same for our businesses to succeed.