• Home
  • Memo
    • Past Memo Archives
    • Podcast (iTunes)
    • RSS Feed
  • Roy H. Williams
    • Private Consulting
    • Public Speaking
    • Pendulum_Free_PDF
    • Sundown in Muskogee
    • Destinae, the Free the Beagle trilogy
    • People Stories
    • Stuff Roy Said
      • The Other Kind of Advertising
        • Business Personality Disorder PDF Download
        • The 10 Most Common Mistakes in Marketing
          • How to Build a Bridge to Millennials_PDF
          • The Secret of Customer Loyalty and Not Having to Discount
          • Roy’s Politics
    • Steinbeck’s Unfinished Quixote
  • Wizard of Ads Partners
  • Archives
  • More…
    • Steinbeck, Quixote and Me_Cervantes Society
    • Rabbit Hole
    • American Small Business Institute
    • How to Get and Hold Attention downloadable PDF
    • Wizard Academy
    • What’s the deal with
      Don Quixote?
    • Quixote Wasn’t Crazy
      • Privacy Policy
      • Will You Donate A Penny A Wedding to Bring Joy to People in Love?

The Monday Morning Memo

Have You Misinterpreted the Data?

February 12, 2018

| Download
https://podcasts.captivate.fm/media/643ecb07-ceae-4bb7-b32d-52658bacd0ae/MMM180212-MisinterpretedData.mp3

“The data is conclusive,” he told me, “our close rate is much higher when customers call us on the telephone instead of going to our website. Therefore, you need to write ads that drive customers to the telephone.”

“I agree that the data is conclusive,” I told him, “and it says you need to fix your half-assed website.”

The research community has embraced a new buzzword. They take great delight in demanding that everything be “evidence based.” It’s a little like listening to a parrot: “Evidence based.” “Evidence based.” “Evidence based.” “Evidence based.”

By themselves, these two words seem harmless. After all, every new idea is based on evidence. But the smug and devilish side of this trend toward “evidence-based” methodology is that the phrase has come to mean “scientific, conclusive, and therefore above debate.”

In other words, if you want everyone to shut up and swallow your recommendation, all you have to do is raise your voice and announce that it is “evidence based.”

I think they learned this trick from online marketers. (Lest the broad brush of that statement paint innocent people with a fault that is not their own, allow me to say that I know several brilliant, online marketers who gather data responsibly and examine it from every perspective. They agree that numbers can whisper opposite statements when viewed from different angles.*)

I’ve never seen anyone make a decision that wasn’t based on evidence.

So the question isn’t whether you’re basing your decisions on evidence. Of course you are. The question is whether you’re interpreting that evidence correctly.

Here’s how I explain this cognitive bias that has become so alarmingly evident: “The intellect can always find logic to justify what the heart has already decided. Consequently, data is often used in the same way that a drunk man uses a lamppost; for support, not for illumination.”

Let’s examine the facts.

FACT: The Japanese eat very little fat and suffer fewer heart attacks than the British or Americans.
FACT: The French eat a lot of fat and suffer fewer heart attacks than the British or Americans.
FACT: The Japanese drink no red wine and suffer fewer heart attacks than the British or Americans.
FACT: The Italians drink lots of red wine and suffer fewer heart attacks than the British or Americans.
FACT: The Germans eat sausages with beer and suffer fewer heart attacks than the British or Americans.
EVIDENCE BASED CONCLUSION: Eat and drink what you like. It’s speaking English that kills you.

The misinterpretation of data is as old as humanity. “Post hoc, ergo propter hoc,” is the ancient Latin name for this most common fallacy of logic. It means, “the second thing followed the first thing, therefore the first thing caused the second thing.”

But correlation rarely indicates causation.

Another limitation of data is that it cannot tell you the right thing to do. It can only tell you the result of what you have already done.

Am I against data? Of course not. Data is information, and information is powerful.

But like all powerful things, it can hurt you if mishandle it.

Five safeguards you should use when evaluating data.

Ask:

1. What were the methods of data collection?
2. Could those methods have influenced the findings?
3. Is there any other way to look at these numbers? (i.e. – Are they saying “drive customers to the telephones,” or are they saying “fix the website”?)
4. Is there a chance the persons who prepared this information have a bias or an agenda?
5. If the data reveals a surprise, is that surprise supported by indicators outside the data?

You’ve heard it said that “numbers don’t lie.” I’ve heard that, too.

But I also remember my grandfather Roy looking at me after a data-quoting salesman had walked away. He said, “Little Roy, never forget: figures lie when liars figure.”

Granddad, it’s been fifty years.

I never forgot.

Roy H. Williams

* Jeffrey and Bryan Eisenberg, Ryan Deiss, Cedric Yau, Michele Miller, Sarah Klenke, Austin Leonard, Robert Brace, Paul Boomer, (and all the Wizards of Ads.) 🙂

FAILURE ALERT: Xerox, Dr. Pepper Snapple, Buffalo Wild Wings and dozens of other brand-name companies have failed to stay relevant in our fast-changing world.  Consequently, they have ceased to survive as independent companies and are now mere possessions of stronger, more flexible firms. Some of yesterday’s superstars like Toys R Us are closing up shop altogether. And the forces that are affecting those big companies are affecting your company as well. Do you know how to stay flexible and relevant? Listen in as Allen Adamson shows roving reporter Rotbart exactly what it takes to stay flexible, relevant, independent, and strong. The place to be, the place to see, is MondayMorningRadio.com

 

Email Newsletter

Sign up to receive the Monday Morning Memo in your inbox!

Download the PDF "Dictionary of the Cognoscenti of Wizard Academy"

Random Quote:

“Of all that is written, I love only what a person has written with his own blood.”

- Friedrich Nietzsche

The Wizard Trilogy

The Wizard Trilogy

More Information

  • Privacy Policy
  • Wizard Academy
  • Wizard Academy Press

Contact Us

512.295.5700
corrine@wizardofads.com

Address

16221 Crystal Hills Drive
Austin, TX 78737
512.295.5700

The MondayMorningMemo© of Roy H. Williams, The Wizard of Ads®