Team Data

 

Building a Data-Driven Optimization team

 

@guido | x@gui.do

Conversion Rate Optimization (CRO)

 

Where do we start?

What tools do we need?

How do we grow(th hack)?

How do we prioritize tests?

 

Who's going to do this?

Know...

 

... your maturity level

... the Website Optimization Workflow

... how to prioritize

... who to hire

 

Necessary components

 

Platform

People

Process

 

But first:

What is your maturity level?

Digital Analytics Maturity Model (DAMM)

by Stéphane Hamel

Maturity Levels

Level 1

Level 2

Level 3

Level 4

Level 5

Visitors, Visits, Views

Top 10s

Demographics

Technographics

Top entry/exit

Performance

Capacity & Security

 

Path analysis
Funnel reports
A/B testing
Campaign Optimization
KPIs
Dashboards

 

Omnichannel                    aggregation
External Data Analysis
Segmentation
Segment testing
Behavioral Targeting

 

Profiling &   Personalization
Cost-shifting analysis
Lifetime value
Big Data Targeting
Process analytics

 

Real Time Decision Targeting
Activity Based  Cost
Strategic planning
Full omnichannel integration
Predictive Analytics
Balanced Scorecards

DAMM by Stéphane Hamel

Maturity Levels

Remember the Maturity Levels!

Self assessment:
DigitalAnalyticsMaturity.org

The Platform

 

The Platform

 
External Internal
External Outsource xxx
Internal SaaS Tool Buy/Build yourself

Tooling

People

Choosing the Ideal Solution

 

Testing

Adding a new software tool

 

The People

Teams

Level 1

Level 2

Level 3

Level 4

Level 5

Single person

One person leading optimization efforts accross teams/departments

 

Single team focussed on online optimization

 

 

Integration with teams from all channels/ departments

 

 

Multiple agile CRO teams

+

specialized umbrella teams

These are not separate stages,

it's a continuum

Team Experts

(Almost) all levels:

The Executive Sponsor

 

 

 

 

aka

Shit Umbrella

Level 1: the "lone nut"

No CRO job title

Really enthusiastic about the possibilities

Level 2

 

CRO Master /

Growth Hacker

UI/UX Designer

Frontend

Developer

CRO Master / Growth Hacker

 

Source: GrowthTribe Academy

Level 3

 

Traffic

UX Research

Content

Specialist

Tester

Level 4

 

Data

Analyst

Data

Specialist

Behavioral Analyst

Level 5

 

Multiple

Teams

Team

managers

Data visualization specialist

Statistician

Teams

Level 1

Level 2

Level 3

Level 4

Level 5

The lone nut

CRO Master

UX/UI Designer

Frontend Developer

 

UX Researcher

Content specialist

Tester

Data Analyst

Data Specialist

Behavioral Analyst

 

CRO Team managers

Data visualization specialist

Statistician

Roles can be combined

(especially in the beginning)

Supporting Departments

 

Core

team

Business

Sponsor (shit umbrella)

Customer Support

Traffic team

Marketing

(Backend) Development

The Processes

 

Input

Prioritization

Testing

Reporting & Deployment

Input

Prioritization

Testing

Reporting & Deployment

Business Intelligence

General Research

Corporate Needs

User Research

Product Roadmap

Product Changes

Brainstorm

Test Results

Idea list

Input

Prioritization

Testing

Reporting & Deployment

Why?

What?

Input

Prioritization

Testing

Reporting & Deployment

Input

Prioritization

Testing

Reporting & Deployment

Hypothesis

Set test parameters
& variants

Create

Deploy

Analysis

Input

Prioritization

Testing

Reporting & Deployment

Reporting

  • # Participants/ variant
  • Test duration
  • Significance level (p-value)
  • Confidence interval
  • Effect size
  • Power
  • # test repetitions

Deployment

Calculate ROI


Worth it?

Yes > Forward to devteam

No> Optimize/Retry/Discard

    Input

    Prioritization

    Testing

    Reporting & Deployment

    Optimization Cycles

    1 2 3 4 5 6 7
    Test A Research Design/
    Develop
    Q&A / Approval Live Live Analysis Share/ document
    Test B Research Design/
    Develop
    Q&A / Approval Live Live Analysis
    Test C Research Design/
    Develop
    Q&A / Approval Live Live
    Test D Research Design/
    Develop
    Q&A / Approval Live
    Test E Research Design/
    Develop
    Q&A / Approval
    Test F Research Design/
    Develop
    Test G Research
     

    Week numbers

    People & Processes are

    Crucial

    to your Continued Success

    How will you benefit?

    • Shorten meetings by avoiding deliberations

    • Avoid interpersonal conflict by putting differing opinions to the test

    • Empower all members of the team to optimize their work

    • Align teams around shared goals, rather than conflicting ones 

    Experiments in any stage of the funnel

     
    • Sales 

    • Marketing 

    • Product 

     

    Team Data: Building a Data-Driven Optimization Team

    By slidescom@gxjansen.com

    Team Data: Building a Data-Driven Optimization Team

    Presentation for Optimizely Experience Amsterdam 2015

    • 1,359