Preregistration
Open Science Bites
The
Replication Crisis
https://xkcd.com/882/
https://projects.fivethirtyeight.com/p-hacking/
Questionable Research Practices?
Munafò, et. al. „A manifesto for reproducible science“. Nature Human Behaviour
Anything can become significant with a
creative approach
- Optional stopping (ending data collection when p<0.05)
- Selective reporting of variables and conditions
- Exclusion of participants („Outlier“)
- Conducting tons of analyses but only report the one that best fits the hypothesis (“Garden of forking paths”)
- Adding covariates to the analysis
- Transforming the data
- Forming hypotheses post-hoc (HARKing)
https://osf.io/q2ekw
The Solution:
Preregistration!
What is Preregistration?
The specification of a research design, hypotheses, and analysis plan prior to observing the outcomes of a study
Why Preregister?
- Transparency: Differentiate between confirmatory and exploratory hypotheses
- Avoid HARKing
- Minimize false positives through p-hacking
- Think more deeply about hypotheses/design in advance
- ...?
What to preregister?
As much as possible!
- sample size, hypotheses, analysis...
- decision rules etc.
- standards
How to preregister?
- aspredicted.org
- osf.io
- RTG study checklist! :)
Aspredicted
OPEN SCIENCE FRAMEWORK
What is a good Preregistration?
(next time?)
Title Text
-
Have any data been collected for this study already?
-
What's the main question being asked or hypothesis being tested in this study?
-
Describe the key dependent variable(s) specifying how they will be measured.
-
Anything else you would like to pre-register? (e.g., secondary analyses, variables collected for exploratory purposes, unusual analyses planned?)