Thanks!

@saltnburnem

@saltnburnem

@saltnburnem

#BeerCityCode | #DeveloperCommunity

#BeerCityCode

@saltnburnem

#BeerCityCode | #DeveloperCommunity

#DeveloperCommunity

@saltnburnem

#BeerCityCode | #DeveloperCommunity

Chris DeMars

Developer Advocate

@saltnburnem

@saltnburnem

#BeerCityCode | #DeveloperCommunity

FREE AGENT!

  • International Speaker
  • Love spooky stuff
  • Tattoo collector

About Me

@saltnburnem

#BeerCityCode | #DeveloperCommunity

https://www.twitch.tv/saltnburnem313

@saltnburnem

@saltnburnem

#BeerCityCode | #DeveloperCommunity

Demystifying

the

Dreaded A/B Test

@saltnburnem

#BeerCityCode | #DeveloperCommunity

It's not that scary

trust me...

@saltnburnem

#BeerCityCode | #DeveloperCommunity

What is A/B testing?

@saltnburnem

#BeerCityCode | #DeveloperCommunity

What is A/B testing?

A/B testing, otherwise known as split testing, is the process of testing two different versions of a web page or product feature against the original, existing version.

- Talia Nassi

@saltnburnem

#BeerCityCode | #DeveloperCommunity

Username

Password

Login

provider

Create Account

A

@saltnburnem

#BeerCityCode | #DeveloperCommunity

Username

Password

Login

provider

Create Account

Username

Password

Login

provider

Create

A

B

@saltnburnem

#BeerCityCode | #DeveloperCommunity

Why?

@saltnburnem

#BeerCityCode | #DeveloperCommunity

⬇️ Answers ⬇️

  • Awesome user experience
  • Increase revenue
  • Data correlation for better business decisions
  • Fine grained control of code exposure

@saltnburnem

#BeerCityCode | #DeveloperCommunity

How?

@saltnburnem

#BeerCityCode | #DeveloperCommunity

Well set up an experiment of course!

@saltnburnem

#BeerCityCode | #DeveloperCommunity

That is what we call a moron.

That is not an experiement.

@saltnburnem

#BeerCityCode | #DeveloperCommunity

First

@saltnburnem

#BeerCityCode | #DeveloperCommunity

First

Gather that data!

@saltnburnem

#BeerCityCode | #DeveloperCommunity

❗️ Baseline before experiment! ❗️

@saltnburnem

#BeerCityCode | #DeveloperCommunity

@saltnburnem

#BeerCityCode | #DeveloperCommunity

Scientific Method

@saltnburnem

#BeerCityCode | #DeveloperCommunity

Scientific Method

  1. Ask a question
  2. Do research
  3. Construct a hypothesis
  4. Experiment
  5. Draw conclusion

@saltnburnem

#BeerCityCode | #DeveloperCommunity

Scientific Method

#HalfStackConf | #DeveloperCommunity

@saltnburnem

  1. Construct a hypothesis
  2. Experiment

If we ship this new button, our retention will increase by 20%.

Hypothesis

@saltnburnem

#BeerCityCode | #DeveloperCommunity

Experiment time!

@saltnburnem

#BeerCityCode | #DeveloperCommunity

Username

Password

Login

provider

Create Account

Username

Password

Login

provider

Create

A

B

Control

Treatment

@saltnburnem

#BeerCityCode | #DeveloperCommunity

Feature Flags

@saltnburnem

#BeerCityCode | #DeveloperCommunity

Segment Users

@saltnburnem

#BeerCityCode | #DeveloperCommunity

Username

Password

Login

provider

Create Account

Username

Password

Login

provider

Create

A

B

Control

Treatment

50%

50%

@saltnburnem

#BeerCityCode | #DeveloperCommunity

Flex that UI muscle!

@saltnburnem

#BeerCityCode | #DeveloperCommunity

You can be a CSS dev and use feature flags!

@saltnburnem

#BeerCityCode | #DeveloperCommunity

CSS...like CSS

IYKYK

@saltnburnem

#BeerCityCode | #DeveloperCommunity

// get treatment
  let treatment = client.getTreatment('css_demo');

  // get features to flip based upon CSS classes
  const featureA = document.querySelectorAll('.a');
  const featureB = document.querySelectorAll('.b');

  if (treatment === 'on') {

      featureB.forEach(element => {
          element.classList.add('hide')
      })
  } else if (treatment === 'off') {
      featureA.forEach(element => {
          element.classList.add('hide')
      })
  } else {
      console.error('Error connecting to Split')
      // default rule here
      featureA.forEach(element => {
          element.classList.add('hide')
      })
  }

});

@saltnburnem

#BeerCityCode | #DeveloperCommunity

Demo

@saltnburnem

#BeerCityCode | #DeveloperCommunity

It's a happy cry!

Resources

https://bit.ly/css-feature-flags

@saltnburnem

#BeerCityCode | #DeveloperCommunity

Chris DeMars

@saltnburnem

@saltnburnem

#BeerCityCode | #DeveloperCommunity

Demystifying The Dreaded A/B Test

By Chris DeMars

Demystifying The Dreaded A/B Test

  • 74

More from Chris DeMars