Pitfalls in data

and how to avoid them

Maarten Lambrechts

Pitfalls in

metadata

statistics

visualisation

 

PITFALLS

Metadata

"... in Brussels, nearly 62 percent is of foreign origin"

Metadata = data about the data

Collected by whom?

Collected how?

Collected why?

Collected when?

Definitions?

Metadata

 

Determine validity of conclusions

Describe limitations on the use of the data

Determine comparability

Without correct units numbers are meaningless

 

PITFALLS

Statistics

Percentages  & percentage points

"2.1% of mask wearers tested positive for COVID-19, versus 1.8% of non-mask-wearers. The difference is only 0.3%."

This is an difference of 0.3 percentage points

Or:

(2.1 - 1.8)/2.1 = 14% decrease in positive tests

% - % = percentage point

(new - old)/old = % change

That's not normal

Top 5 EU electricity consumers

Country
 
1. Germany
2. France
3. UK
4. Italy
5. Spain
Electricity consumption (Gwh)
517.377
442.372
303.903
286.027
232.515

Congrats: a population ranking!

Please divide by the number of people

Top 5 EU electricity consumers

Country
 
1. Iceland
2. Norway
3. Finland
4. Sweden
5. Luxembourg
Elektricity consumption (Mwh/cap)
49,7
21,5
14,7
12,6
10,6

Especially relevant for maps

Make numbers comparable

(=normalising):

per capita, per surface area, ...

Mean vs median

How many soulmates do you think a person can have?

The median?

Rank the data: the median is the middle value

The median is less sensitive to outliers. Use it!

Distributions

Summary statistics rarely describe somebody's lived experience and never ring true for the whole population

Data are much more than averages

Use distributions whenever you can

Correlation vs  causality

Correlation. Is. Not. Causation.

Confidence

intervals

"The margin of error is  3,2 percent."

Uncertainty is inherent to survey results. Keep the margins of error in mind

Big & small chances

People who ate 76 grams of red and processed meat per day had a 20% higher chance of developing colorectal cancer compared to others, who ate about 21 grams a day.

Of 10.000 people in the study who ate 21 grams of red or processed meat each day, 40 developed colorectal cancer. Among those who ate 76 gram per day, 48 did so.

21 grams => 40/10.000 = 0.4%

76 grams => 48/10.000 = 0.48%

+20%, or +0.08 percentage point

+20% of a small chance is still a small chance

The researchers said B.1.1.7 led to 227 deaths in a sample of 54,906 patients. That compares with 141 deaths in roughly the same number of patients who were infected with other strains.

Earlier strains:  141/54906 = 0,26%

UK strain: 227/54906 = 0,41%

 

But still 86 more deaths

But relatively small differences can be meaningful

Apples & oranges

Compare

regions to regions

countries to countries

apples to apples

oranges to oranges

Exponential growth

Exponential

skyrocketing

Percent & percentage points

That's not normal

Mean vs median

Distributions

Correlation vs causality

Confidence intervals

Big & small chances

Apples and oranges

Exponential growth

PITFALLS

Visualisation

Keep the pies for dessert

Don't cut bars

Respect proportions

Don't cut time axes

Compare apples to apples

Scale circles by surface area

Don't do 3D

Double the axes,

double the mischief

Every map is a lie

Keep the pies for dessert

Don't cut bars

Respect proportions

Don't cut time axes

Compare apples to apples

Scale circles op by surface area

Don't do 3D

Avoid double axes

Every map is a lie

Thanks!

slides.com/maartenzam/pitfalls-mediahuis