Two of these are true

One was made up. All three are the kind of thing you might act on. Pick the fake.

round 1 0 right / 0 played

Notice how little the fake had to change to pass. That's the trick: a false claim borrows the shape of a true one — a plausible source, a confident number, a rule that sounds like it must exist. The cost isn't being wrong in a quiz. It's the decision you make afterwards.

How a generalization is born

How a generalization is born

The end. Sort of.

It doesn't stop on its own. Someone has to ask: says who?

believers: 0 start

More Info

A statistic is a number that's meant to represent a group of people. They can be important and useful, but are easy to misinterpret and use maliciously. Here's how to spot a stat that has some credibility to it, and how a bad one can become harmful.

63% real people one number
the basics

A number hides a lot

Every stat takes a whole crowd of people and mashes them into one figure. If done well, it can act as a fair summary. But if lazily created, it can become a huge misinterpretation.

So don't start with "what's the number." Start with "how was the number determined."

asked 4 one loud voice swings everything asked 400 one voice barely moves the needle
sample size

How many people did they actually ask?

Ask four people how tall they are and you might get a completely different data set than if you asked four hundred. A small sample set isn't lying, it just doesn't speak for everyone.

And if a stat won't tell you how many people it's based on, that's usually because the answer would affect your perception of it.

sampled one corner, spoke for the whole room
sampling bias

Who they asked decides what they hear

If you wanted to figure out the most well-percieved phone, you wouldn't exactly only collect data from Apple Store customers. Many sampling biases exist, and all of them can affect a statistic's accuracy.

A solid poll pulls a random mix of the whole group. Focusing on a smaller collection within that group doesn't allow you to connect it to other people.

ice cream sales shark attacks ≠
correlation vs causation

Rising together isn't the same as causing

Ice cream sales and shark attacks both go up at the same time every year, but they aren't exactly correlated in any way. When you don't consider confounding variables, like seasonal trends, your data can become more of a conspiracy theory than an actual connection.

So when you hear "X is linked to Y," stop and ask what else might explain it.

looks huge starts at 48 same data starts at 0
misleading graphs

Check where the axis starts

The same two numbers, 49 and 51, can look like a marginal gap or a small difference depending on where the bottom of the chart sits. Start the axis near the top instead of at zero and a tiny gap suddenly looks like a cliff.

Before you panic at a dramatic bar chart, look at the numbers running up the side. Half the time the picture is doing all the work.

+100% sounds terrifying became 1 case turned into 2
percentages without context

"Doubled" can mean almost nothing

A "100% increase" might just be a disease going from 1 case to 2. Percentages leave out the starting number for a reason. "Up 100%" makes people nervous. "One more case than last year" doesn't.

Whenever you see a big percent, check for the actual count. Out of how many? Up from what?

quick gut check

Five questions before you believe it

  • Who's saying it, and do they win something if you're convinced?
  • How many people or cases is it really based on?
  • Who got asked, and who didn't?
  • Is there an actual source you can click, or does it just say "studies show"?
  • Would you still repeat the headline once you've read the rest?
one person "those people" "everyone knows"
how it becomes a stereotype

One story, blown up until it's "everyone"

Untrue statements can quickly become malicious. There doesn't need to be substantial proof for a claim to be widely believed.

A shaky stat is where a lot of stereotypes borrow their fake credibility. "It's not prejudice, it's statistics" only works if nobody bothers to check the statistics.

This doesn't mean every number is out to get you. Good statistics are one of the best tools we've got for actually understanding stuff. Just be aware of how a small error in a study can produce a completely innacurate result.