Sharing experiments between products

Let's start with quick intro

When do we create an experiment?

Every time we change user experience

 - and -

Every time we risk user experience

How does client know the experiment state?

First option - Lazy

  • Notice that spec is only identified by name. Be responsible, namespace.
  • Lazy = Slow
  • Lazy = Async

Second option - Bulk

  • Bulk = Premature
  • Exposure id helps
  • Multiple scopes help
  • Ajax = Overhead
  • Ajax = After render

Third option - Render bulk

<script>
  window.__EXPERIMENTS__ = <%- experiments %>;
</script>

 

  • Bulk = Premature
  • Exposure id helps
  • Multiple scopes help (lazy loaded)
<script>
  window.__NEW_THEME__ = <%- newTheme %>;
</script>

 

Don't, Don't, Don't. Plz Don't

How do we define new experiment?

  • npm install -g petri-specs
  • petri-specs new

New stack

Old stack

  • npm install -g generator-wix-angular
  • yo wix-angular:experiment specs.blah

Deploy

  • git commit
  • git push
  • rc + ga
  • petri sidekick / guineapig

Do I really need to GA?

  • git commit
  • git push
  • wait for build to finish in teamcity
  • petri-specs update

No RC required, although you probably need RC anyway for version override

Use static override extension.

Use petri sidekick extension.

Use elgar (vpn required)

plz plz plz don't do hackish ad-hoc solutions

So what happens if experiment is shared?

  • Make ajax call to load it lazily?
  • No!!!!
  • Conduct it on server and add to rendered experiments?
  • No!!!!
  • Do this?
  • OMG.
  • No!!!!
  • All we need is simply to define multiple scopes, that's it.
  • Let's see.

Experiments 101

By Shahar Talmi

Experiments 101

  • 1,676