How to build
on day one
4 steps before every experiment
Your first experiment ideas
Is it done yet?
Let’s go back and
look at the basics…
4 Steps before every experiment
Define your goals
- Form Submits?
- Outline your hypothesis
- Define your test subjects / target audience
- Calculate your sample size
Ideas… keep ’em coming!
Do you have a list / backlog of testing ideas?
Don't have ideas already?
Usability testing / hallway tests
Chickity Check ya test
wreck your test
- Variation Code - is it appearing and working correctly?
- Timing - Does your variation show up when it should, without flickering?
- Targeting - Is your variation on the right page, for the right audience?
- Goals - Are your goals triggering when they should?
Is it Done yet?
This team has data ,
Ryan Meerstein, Targeted Victory
Just an Original, No variation
w/ Main Goal
Run for 7 / 14 / 30 Days
Goal: Estimate sample size potential, get baseline conversion
Enter data from Baseline Experiment
Pick a realistic Sample Size
Goal: Define sample size for future experiments
& Wait ;)
Build and start Experiments
Look at Experiments when Sample Size
reached, not before
Goal: Get actionable insights that aren't false positives
You've collected results
that you can be confident about!
How long do i run the test for?
(Sample size x 2) / Average visitors per day = Estimate no. of days to run test.
Slides will be available on slideshare
For any feedback/questions/resources contact me:
Brighton CRO Presentation