banner.png

The fallacy of data-driven management, why yes-men exist and other lessons in politics

Posted by Arjun Moorthy

Nov 23, 2013 9:28:00 AM

Recently a colleague made a thinly veiled accusation that I was biased about an analysis I was working on and was "spinning the data."  The allegation hurt deeply because I have never consciously cooked any data to match an outcome I wanted and I sincerely believed I was working towards the truth.  I went through much of the day distraught at this attack on my credibility but a series of discussions afterwards at home helped me learn three important lessons that included some surprising conclusions.

 

moneyball

Lesson 1

 

When I got home a number of family friends had come over for dinner.  One of the guests was a well respected professor and we got to talking about how an MBA doesn't really prepare one for the political nature of real life and yet learning to navigate this is a greater indicator of success than any academic learning one may get at school.  

The professor relayed how, after he saw the transient nature of colleagues (i.e. most get canned at some point), and the arbitrary nature of many work decisions, he came to the conclusion that work is truly a means to an end and stopped getting emotionally involved with his work.  He said he maintains a separate work life and personal life and never interacted outside the workplace with any of his colleagues.  

I'd heard of such approaches before from others (some change clothes when they come home just to physically switch off) but they seemed unnecessarily draconian yet here was a tenured professor, with no risk of losing his job and arguably at the top of his field, saying it was the only way he could function.  If so why was I taking my job so personally, risking my sanity and employment on my definition of the "truth"? 

I seem to have to re-learn this lesson every few months - to not take things so personally and to realize that everyone is speaking from their point of view.  My sales coach, Rick, has taught me - everyone has an agenda and no one should be trusted [blindly].  I used to think this was absurdly pessimistic but the more I interact with humans the more I realize this is just a fact (and true even of me) so no point brooding over it.

Lesson 2

 

The professor and I then talked about a famous case we both did at business school where the story centered on a race car team and whether they should race the next day or not given a set of data about possible malfunctions with the engine.  The case is detailed and provides much context on the team's history, the car's history, the driver's history, the competitive environment etc.  By the end nearly all the readers were in favour of racing the car the next day.  

During the class discussion the teacher reveals that the case is really about the Challenger space shuttle and by choosing to "race" we, the readers, had just sent 7 men and women to their death.  Many of us, particularly me, were hit hard by the conclusion and realized that we saw in the data exactly what we wanted to see - that we were a team with a mission and could not possibly stop once down that path.

Talking about the case so many years later reminded me of a recent article in the Economist where Amgen Pharmaceuticals did a test where they could reproduce only 6 out of 53 of the landmark science studies in cancer research.  Read that again - almost 90% of the most important scientific research couldn't be reproduced by other experts in the field.  Reproducibility is the gold standard of scientific research to validate new findings and yet the most famous research was now being questioned for its basic validity.  There are several reasons for this but the main conclusion was that even scientific experts are not always objective and subconsciously fit the data to match their hypothesis.  

If trained scientists and statisticians, who pride themselves on the truth above all else, are prone to bias (or sometimes just plain errors) then wasn't it possible I was too?  Maybe the allegation was accurate insofar as I may be subconsciously biased.  I always try to do a "sniff test" when I'm studying data to see if the analysis matches with intuition and empirical observations but maybe this was subconsciously fitting data to what I wanted to have happen.  

Nevertheless, the more important extrapolation from the article is that it essentially says that all humans are biased.  And this is where the story gets really interesting.  When you juxtapose this conclusion with the professor's comment about the capricious nature of decision making you realize what drives outcomes in business is not data.  Many of us would like to believe we're data-driven but it turns out we're not that good at analyzing data and decisions are ultimately made more on our gut feel than purely on what the data says.  Data helps but it is by no means the deciding factor. 

And this is why many of us rely on validation from our peers when we're making decisions.  This is also why "yes men" exist in corporations - because it feels good to have your hunch backed up by someone you trust when you have to make so many decisions based on gut.  The upshot of this is that once you trust someone implicitly you are most prone to make a mistake.  As historian Lord Acton said "power tends to corrupt and absolute power corrupts absolutely."


Lesson 3

 

The third and final lesson was how the scientific community tests any new material published.  The professor recounted how his research was bashed at every step as was the normal process with peer reviews (and the seeming parallel to my getting challenged at work constantly).  He commented on how colleagues contradicted him on every point - sometimes with good reason but more often than not to show their own intelligence and possibly because his findings simply contradicted pre-existing research.  i.e. even the brightest folks are interested in protecting their turf and find it hard to re-examine long held beliefs despite data to the contrary.

As I was mulling over this disturbing conclusion I sought out Jeff Pfeffer's book on "Power: Why Some People Have It and Others Don't."  Pfeffer is a professor at Stanford whose class I took years ago and his book has been the most practical of all my learnings at school.  Pfeffer cites a study which concluded that while intelligence is correlated to success it accounts for less than 20% of the variation in work performance over the years.  And "intelligence, beyond a certain level, may lead to behavoiurs that make acquiring or holding on to influence less likely."  i.e. beyond a certain point intelligence hinders success in a corporate setting.  That's a pretty mind-blowing conclusion.  Perhaps instead of aspiring to be smart a genius, or aspiring for our children to be smart geniuses, we should aspire to simply be [smart and] savvy.

Pfeffer does a great job explaining why intelligence can be limiting - it can be seen as intimidating by others, or make the intelligent person less sensitive to others who "don't get it", or simply makes the intelligent person arrogant and prone to error themselves.  

I've blogged about the difference between book smarts and street smarts but it seems this too is a lesson I am doomed to keep re-learning.  Maybe I'm not so smart after all - and paradoxically then maybe there's hope for me.

Topics: politics

Career Advice I Wish I Knew Earlier 

Hello.  I started this blog to distribute some of the best career advice I have been given over the many jobs I've had.  I've been fortunate to work for and with some great bosses like Brian Halligan, Francis DeSouzaNancy Kamei, and Rick Roberge, and some unique companies, like The Boston Consulting Group, that invest heavily in making each employee a success even after leaving the firm.

The advice and training I received here stands in contrast to my experiences with some not-so-great bosses and companies I've also worked for.  I'm continualy amazed at how valuable good advice has been in my career so I hope to pass on the good advice, and insights from mistakes I've made, via this blog. 

Thanks in advance for your comments, particularly when you can improve upon the ideas posted.

Subscribe to Email Updates

Search blog posts

Recent Posts