Going beyond the low hanging fruit


@kylerush






Low hanging fruit





Low hanging fruit experiment:

Recurring donation messaging


Close-up




Close-up





Result:


+110% recurring donations


at 99.9% confidence level

Quick Donate


Quick Donate


$115 million


1.5 million users


10 month lifespan






Guiding users




Experiment:

Log in indicator

Control

Variation




Result:

+9% donations






User experience




Experiment:

Left aligned labels



Control                     Variation



Result:

Statistical tie





Experiment:

Two column layout









Result:

Statistical tie





Experiment:

Labels within inputs

Control                  Variation




Result:

Statistical tie




Experiment:

"Sequential"


Step 1


Step 2


Step 3


Step 4





Result:

+5% donations


Important lessons:


1. Lower barrier to engagement



2. Engagement builds investment






Data gathering




Google Analytics to the rescue!




Custom event:

_gaq.push(['_trackEvent', 'donate form error', fieldName]);

Donation error report





Experiment:

Required indicator

Step 1: Donate


Step 2: Quick Donate opt-in


Add asterisks



Result:


Reduction in errors


+7 Quick Donate opt-ins



Quick Donate users


Donate 4x as often


Donate 3X more money


!!!!







Error reduction frenzy





User testing




Experiment:

Employe & Occupation Hint






Result:

-63% errors




Experiment:

Credit card formatting




Result:

-15% errors




Experiment:

Fun backgrounds

Control & variation step 1

Variation step 2

Variation step 3

Variation step 4





Result:

Statistical tie




Experiment:

"Boss" design





Result:

+5% donations





Experiment:

"Choice" design







Result:

Statistical tie




Performance


Initial architecture




Problems


  • No CDN
  • No caching
  • 46 http requests
  • 5 - 7+ second load time

Variation architecture


Performance comparison



webpagetest.org

Slow                                 Fast



Result:


+14% conversions


$32 million





@kylerush