Can the numbers ever lie?

Have you ever accepted a credible statement that was backed by numbers only to have the sneaking suspicion that you have been misled?

Perhaps you are in a position where you need to back up a spurious claim?

Use numbers.  Statistics, facts, figures and correlations.  Make them impressive by using graphs, charts and tables.  Be sure to add some asterisks and attribute sources.  Make the attributions as tiny as possible.  Don’t worry about scales or defining your axes, its visual impact you are going for.

After all, nobody can argue with the hard facts.  The most impenetrable arena of empirical evidence?  Numbers.

Numbers don’t lie.  Unless they do.

From outright lies to conscious misdirection, the misuse of numbers is the trick of the persuader with an unsubstantiated argument.  We have to look closely and question bravely to uncover the truth behind the veil of numerically embellished false credibility.

There are three kinds of lies; lies, damned lies and statistics.

-Benjamin Disraeli

I recently shared a discussion of Daniel Levitin’s book, A Field Guide to Lies in the Information Age.  Levitin explores examples of misinformation and misdirection in the media.  Lies that have stood uncontested, partly due, according to his thesis, to our impaired capacity for critical thinking in the age of information overload, soundbites and device driven news.

In support of critical thinking, this article will explore 6 of the most common ways numbers lie.  (Or at least how numbers are used to drive false conclusions.)

 

6 Ways Numbers Lie

1.  The Small Sample Size

When conclusions are drawn from a very small sample size, whilst they may be true, are not robust.  Such conclusions should not be taken as fact and should not be extrapolated to support further argumentation.

Here is an explanation from Darrell Huff, author of How to Lie with Statistics, where he refers to this form of statistical slight of hand as ‘The Little Figures That Are Not There’.

How results that are not indicative of anything can be produced by pure chance—given a small enough number of cases—is something you can test for yourself at small cost. Just start tossing a penny. How often will it come up heads? Half the time of course. Everyone knows that. Well, let’s check that and see…. I have just tried ten tosses and got heads eight times, which proves that pennies come up heads eighty percent of the time.”

-Darrell Huff, How to Lie with Statistics

How many dentists were asked if they would recommend chewing gum?  Four out of five seems number worth questioning…

 

2.  Crafty Time Frames and Visual Manipulations

One way to make your numbers tell the story you want, whilst ethically questionable falls short of outright fabrication of data.  You could use a method famously used by Tim Cook, CEO of Apple Computers reporting on the sales of Apple iPhone in September 2013.

What’s wrong with this picture:

iphone-sales

 

 

 

 

 

What is wrong is that there is no scale and it is not clear if sales are in the thousands, millions or billions.  Perhaps what is most wrong with it is that it borrows credibility from the undeniable facts, demonstrated in numbers but fails to provide any explanation for what the numbers mean.

To be fair to Mr Cook, this is a powerful, clean, minimalist image.  It is on point, reinforces his message and certainly reinforces the belief of everyone in the room.  It’s just not an accurate reflection of the truth.  Here is an overlay of iPhone sales by quarter published by Quartz – the juxtaposition reveals the reality and the misdirection.

adjusted quarterly sales

Mr Cook seems to have used a misleading set of numbers (cumulative sales) to demonstrate positive sales.  The overall impression is positive, the eye does not register the dip at the top.  The overlying bar graph present a more accurate, but less palatable, view of sales results.

 

3.  Correlation not Causation

When a statistical test shows a correlation between A and B, there are multiple possibilities:

  1. A causes B.
  2. B causes A.
  3. A and B are both caused by a third factor, C.
  4. B is caused by C which is correlated to A.

The misuse occurs with possibilities 3 and 4.  The 4th is the most potentially problematic.  Rather than further investigating the chance correlation a causal link is purported.  An irrelevant argument for a spurious claim.

number correlation causation

4.  Infographics- More graphic than info

The rise of the infographic helps us absorb information quickly – at a glance we can form of view.  We squint slightly then nod our acceptance, satisfied in the knowledge the the numbers have been crunched.

Take a look at this infographic from NSW Health showing the increase in the number of nurses hired.

nsw health nurses

 

Four stick people represent 43,000 nurses – so why are 28 more stick people used to represent an increase of just 3,000 nurses?

That’s a 700% infographic explosion to show a 7% increase.

 

5.  Mutating Big Numbers

Round numbers are always false.

-Samuel Johnson

How to Lie with Statistics was written by Darrell Huff back in 1954.  But even in this age of big data, his cautionary words remind us to remain critical when presented with enormous figures.  Huff observed that “without writers who use the words with honesty and understanding and readers who know what they mean, the result can only be semantic nonsense.”

In the age of big data we can choose the representation that creates the impression we desire.  Use the mean rather that medium, select national aggregates rather that per capita average, or simply blind our audience with a huge impactful number in hopes of eradicating any potential challenge by its sheer intimidating scale.

Joel Best unpacks the the example of 1 million missing children, cited in the US media in the 1980’s but still claimed on the internet today.  The number was highly misleading.  About 800,000 were reported as ‘missing’, but on further investigation it has been proved that these were primarily  short-term disappearances, dominated by teenage girls spending time with their boyfriends without parental permission. The number in ways and in contexts that implied that it referred to the abduction and murder of children by strangers.

 

6.  Polls – Opinion or Otherwise

I think the misuse of opinion polls is commonly understood.  When mentioned by politicians, lobbyists, advertisers or spruikers our eyes glaze over.  We all the claim but we quickly skim over, waiting for some actual evidence.

However, despite public lambasting (a classic from Homer Simpson below) we continue to endure such useless application of numbers.

 

So What?

Numbers don’t mislead people.  People mislead people!

There is a fine line between the misuse of numbers to create a false impression and outright lying.  In both cases the intention is to deceive, for the benefit of the persuader.

Why do we allow the falsehood to convince us?  It could be as Levitin argues, that our capacity for critical thinking is impaired by overload of information we all experience in the digital age?  Perhaps we have allowed a laziness in our ability to reason.

Perhaps those of us that are more mathematically challenged should be extra-vigilant of the ways in which numbers can, in fact, lie.