Why you should distrust polls

@maartenzam

Dataharvest | EIJC

Mechelen

May 19 2017

The Poll Trap

Media order survey

Survey company conducts

Media report

Users click, read, watch

Media order survey

...

Everybody

?

No

Statisticians

5 of the problems with polls

Problem I

Polling itself

Problem II

Sample errors

What happened?

Selection bias

Never ever report on public, online polls

Nonresponse bias

Problem III

Survey results are interpreted

Problem IV

Pollster herding

Problem V

Humans suck at uncertainty

So what to do?

Check methodology

In person vs. online

Random sampling?

Sample size

Weighing and other interpretations

Find confidence intervals

Don't report on changes that are well within intervals

Don't use decimals

Visualize results

But show uncertainty

Use averages of polls

Don't jump on a single poll

Be on the watch for herding

There should be variation between polls

Thanks!

slides.com/maartenzam/dh-polls

Why you should distrust polls

By maartenzam

Why you should distrust polls

  • 3,549