stop building
(and talk to people)
Regan Davis
VP of Product, Data @Jellyvision
Makers of ALEX, an interactive cartoon for explaining HR & benefits to employees
Inconvenient Truths
(not Marty Cagan)
Truth 1:
Truth 2:
It typically takes several iterations to get to the point where it actually delivers the expected business value.
At least half of our ideas are just not going to work.
- Not valuable
- Not usable
- Not feasible
Goal: learn fast. and repeatedly.
Building production-quality software
is the most expensive way to test your idea.
what do i use when?
The Opportunity Solution Tree
Desired Outcome
(OKR)
Opportunity
(Problem)
Opportunity
(Problem)
Solution
(idea)
Solution
(idea)
Experiment
Experiment
Desired Outcome
(OKR)
Opportunity
(Problem)
Opportunity
(Problem)
Solution
(idea)
Solution
(idea)
Experiment
Experiment
Double Diamond
Solutions
Problems
Lean Canvas
1
2
3
how to talk to people
people are liars*
*(but like, nice liars)
Ladder by Javi Ayala from the Noun Project
The Ladder of Evidence
Time and Effort
Value of Learning
Ask what they would do.
Ask what they've done in the past.
Ask for specific stories of what they've done in the past.
Ask them to show you what they did.
Simulate a new experience.
–or–
Observe them during a real-life instance.
Live test of a new experience.
–or–
Future
Past
assumptions
actions
Accurate & Predictive
Future
Past
assumptions
actions
Would you be interested in this product?
- They overestimate how much money they will have in the future.
- They overestimate how much time future them will have to use the product.
- And they are genuinely nice people who want to please you.
Future
Past
assumptions
actions
Have you ever paid for a service like this?
- When?
- For how long?
- What were you using before that?
- What triggered the switch from a free service to a paid service?
- Do you still use that service?
- Why did you stop using that service?
- What do you pay/what did you pay?
Humans are rarely stumped.
into
Research Questions
Interview Questions
| Research Questions | Interview Questions |
|---|---|
| What we need to learn. | What we ask to elicit stories that illustrate what we want to learn. |
Outcome
Drive Engagement
- What keeps people engaging today?
- What's a return user look like? Even in their first visit?
- How is that different than a non-return user?
Opportunity
Research Questions
- Tell me about the last time you went to your HSA site.
- Tell me about the first time you used your HSA site.
Interview Questions
- When / where?
- How did it fit in their day?
- How much time they did have?
- What triggered it?
- What interrupted it?
Listen for...
tips and tricks
don't speak
don't explain
bring a friend
let's try it.
Outcome: Increase Engagement
| Research Questions | Interview Questions | Listen for... |
|---|---|---|
| What keeps people engaging today? | ||
| What prevents people from engaging today? |
| Research Questions | Interview Questions | Listen for... |
|---|---|---|
Learnings from first round of interviews:
People stop watching because they run out of things to watch.
Goal for this round: You want to learn how other people choose what to watch next.
b2b techniques
sourcing interviews
people are liars
Text
- Try to build a funnel
- Ask multi-choice questions
- Target behaviors, not just demographics
(for more info: userinterviews.com )
upfront, kind, & honest.
develop themes
Recording
- Record with consent
- Ask to take a photo
- Zoom / Webex
- Otter app for transcriptions
Rainbow Report
Rose, Thorn, Bud
let's try it.
where to next?
Is this a painful enough problem?
Is there enough opportunity?
Would solving this problem be worth investing the next 5 to 10 years of your life?
let's recap
Desired Outcome
(OKR)
Opportunity
(Problem)
Opportunity
(Problem)
Solution
(idea)
Solution
(idea)
Experiment
Experiment
Ladder by Javi Ayala from the Noun Project
The Ladder of Evidence
Time and Effort
Value of Learning
Ask what they would do.
Ask what they've done in the past.
Ask for specific stories of what they've done in the past.
Ask them to show you what they did.
Simulate a new experience.
–or–
Observe them during a real-life instance.
Live test of a new experience.
–or–
upfront, kind, & honest.
questions?
[appendix]
Opportunity
(Problem)
Solution
(idea)
Experiment
- Is it valuable?
- Is it usable?
- Is it feasible?
- Is this iteration producing the desired outcome?
Desired Outcome
MVP, solution interviews
Usability testing, prototypes
Engineering scoping, risk analysis
"live prototype", KPIs, A/B testing, analytics
jobs to be done
driving outcomes
Customers do not do what they say they do.
We refocused from what customers were saying they wanted to what they were actually doing.
If you build a product based on your customer’s ideal self, you might get the initial sale...
but you’ll struggle to engage them, and you’ll churn through customer after customer.
Start by describing
- The outcome we want -- How we’d measure success
-
What we believe is the problem we’re solving and for whom
- The solution we’d build to solve it
Problem
Reduce Churn
(Reduce dollar churn by 10 points)
Outcome
- XXX
Opportunity
- XXX
http://www.extendslogic.com/business/jobs-to-be-done-cancel-interviews/
http://jobstobedone.org/radio/the-mattress-interview-part-one/
product discovery
Deciding what to build.
product delivery
Building it.
product discovery
Deciding what to build.
product delivery
Building it.
Goal: ship fast.
"Agile"
Scrum
sprints
scrum master
velocity
product discovery
Deciding what to build.
product delivery
Building it.
Goal: ship fast.
- Annual releases
- Quarterly releases
- Monthly releases
- Sprint releases
- Daily Releases
- Continuous Integration
"omg we'll never get to..."
we know what good looks like.
product discovery
Deciding what to build.
product delivery
Building it.
Goal: ??
learn fast.
- Are we meeting stakeholder needs?
- Can customers use it?
- Do customers want our solution?
- Are we solving a problem our customers care about?
- Are we driving toward a desired outcome?
This is what good looks like.
"omg we'll never get to..."
(as of 2018)
Problem
Increase HSA Signups
(+10% signups for those on HDHP plan)
Outcome
- do they understand the value of HSA?
- do they have enough money to contribute?
Opportunity
- who are the people who do contribute?
- how are they different than non-converters?
- what value does contributing to HSA bring them?
- how can we get more converters into ALEX?
Prioritize opportunities, not solutions
Problem
Outcome
- I don't understand HSA is part of plan decision.
Opportunity
- How can we get more converters into ALEX?
Solution
- HSA contribution as part of plan selection (Enrollment)
- Reminder messaging (BC, HSA, Enrollment)
Solution
- Change in messaging content
- Reposition ALEX for full HDHP customers
- Targeted Email campaigns
Increase HSA Conversion
(+10% signups for those on HDHP plan)
does the solution deliver on the opportunity?
...in a way that achieves the desired outcome?
Stop Building and Talk to People
By Regan Davis
Stop Building and Talk to People
Building production-quality software is the most expensive way to validate your ideas. This workshop covers the essentials of customer discovery and validating problems and opportunities so that you can focus on building to solve problems that really matter to your customers.
