User Research

CMPT 363

We do not see things as they are.
We see things as we are.
– From the Talmud

Where Are We?

Where Are We?

Types of User Research

Surveys

Card Sorts

Interviews

Contextual
Inquiries

A/B Testing

Analytics

Usability
Testing

Before You Start Any Research Effort

  • Identify the questions you are trying to answer,
    for example:
    • Who?
    • What?
    • Where?
    • When?
    • Why?
    • How?

Topics

  • Interviews
  • Contextual inquiries

Interviews

Here is your first tip:
an interview should be more
like a conversation,
and less like an interrogation

Types of Interviews

  • Structured
  • Unstructured
  • Semi-structured (also called semi-directed)

Structured

A pre-defined set of questions designed to produce shorter answers – often lacking opportunities for true insight

Unstructured

No question script and more open-ended questions
– can produce more insightful results

Semi-structured

A core set of questions with ad-hoc questions for more details
– insightful and more replicable 

Example Semi-structured Interview Questions

  • What apps do you use daily? Can you describe the last time you used that app? 
  • Are these apps used at any specific times or days of the week? If yes, why at those times? 
  • What app do you find easiest to use? 
  • How did you learn to use that app? Oh I see, please tell me more about that… 
  • Why do you find that app easy to use? 

Things Not to Ask About

  • What might be helpful in the future 
  • How feature "x" should be designed 

Interview Process

  • Decide on what you want to learn 
  • Undertake appropriate level of domain research
  • Decide which representative users to interview 
  • Consider ethical issues (e.g. consent, names, etc.) 
  • Decide information recording strategy 
  • Write interview script, including intro, and conduct pilot-test 
  • Dress in a similar manner as participants 
  • Establish rapport and keep focus on participant
  • Start interview with easy warm-up questions 
  • Present questions in a logical sequence 
  • Be sure to thank participants for their time 

Activity: Analyze Interview Video

Developing Interview Questions

  • Avoid long questions 
  • Avoid compound sentences (i.e. > 1 question) 
  • Minimize hypothetical (e.g. would you?) questions 
  • Avoid unfamiliar terminology 
  • Avoid leading questions (e.g. why do you like…) 
  • Minimize use of yes/no type questions 
  • Start with easier What and When questions, then move onto Hows, and finally Whys

The Art of Listening

  • Be comfortable with silence 
  • Give participants time to expand on initial thoughts, and ideally to start telling stories
  • Provide visual or audio cues that you are focused 
  • Do not offer your own opinions or thoughts   

The Power of Why

  • By inquiring why a user does something in a particular way is a very powerful technique to gain insight into user goals 
  • Why type questions can be seen as threatening sometimes, so try to formulate them in a less aggressive manner   

Asking Users Why

  • What prompted you to do that? 
  • What would be the purpose of that? 
  • Could you explain why that needs to be done? 
  • What was your thinking behind that? 
  • Is there an underlying reason for that?   

Keeping the Interview Going

  • To keep the conversation going try 
    • “Uh-huh”
    • “Yes, that’s very interesting”
    • “Oh yes, I see”
  • To encourage continuation/show that you understand try 
    • Paraphrase or repeat back
  • To get more details try 
    • “Tell me more about…”
    • “And what did you do then?”
  • To clarify try 
    • “If I understand, your saying…”
  • To change topics try
    • “Great, now let's next talk about...”
    • “Let's go back to something you mentioned earlier...”

Interview Location

Ideally interviews should occur in the same environment that a person would be using the product in (e.g. workplace, home, etc.)

Activity: Interview Questions

With the focus of learning how your fellow students find supplemental learning materials for their courses, write down 2-3 key questions
that you might ask

Interviews Summary

  • User research methods
  • Types of interviews   
  • Interview process   
  • Developing interview questions   
  • Interview tips 

Contextual Inquiries

What is a Contextual Inquiry?

Observing people in their own environment perform their actual work with a product

Why Perform a
Contextual Inquiry?

  • Who our are users?
  • What are their tasks?
  • What are their values, concerns and issues?

Key Principles

  • Context
  • Partnership
  • Interpretation
  • Focus

Activity:
Share SFU Website Stories

In groups of two or three, swap stories about
using any of SFU's websites

Possible Methods

  • Work-based interview
  • Post-observation
  • Artifact walkthrough (most common approach)
  • Performance-based focus group

Inquiry Plan Outline

  • Overall focus
  • Specific concerns
  • Participant demographics
  • Inquiry method
  • Inquiry schedule
  • Pre-inquiry survey
  • Inquiry script, including core interview questions

Inquiry Script Outline

  • Introduce yourself
  • Restate focus of session
  • Set session expectations
  • Review confidentiality of information
  • Get permission to record (if needed)
  • Ask participant interview questions
  • When appropriate, “transition” to CI (contextual inquiry) mode and ask participant to demonstrate tasks

Example Interview Questions
for a New Product

  • Could you describe your current job/role?
  • Can you describe a typical workday or week?
  • What makes a good or bad workday?
  • What activities do you most frequently perform?
  • What activities are most important to complete?
  • Are there any activities that you would like to change the workflow of? If so, why?
  • Are there any activities you would like to reduce or eliminate? If so, why?

Example Interview Questions
for an Existing Product

  • Could you describe your current job/role?
  • Can you describe a typical workday or week?
  • On average, how often do you use <product>?
  • What are your main goals (reasons) for using <product>?
  • What is your general opinion of <product>?  What are its strengths and weaknesses?
  • What is an example of a key task that you use <product> for?
  • How often do you perform that task? Please demonstrate
    that task.

Probing for More Information...

  • Tell me a little more about...
  • What are your thoughts about...
  • Describe a bit more about...
  • Share some more about...
  • Talk some more about...
  • Help me understand a little more about...

Time for Questions About
Contextual Inquiries

  • What we’ve covered in this section
    • Key principles
    • Possible methods
    • Inquiry plan and scripts
    • Interviewing long-time customers
    • Probing for more information
  • Coming up
    • Session tips

Session Tips

  • Perform your own domain research before the sessions
  • Ensure to allocate enough time for each session, and between sessions
  • If participant want to vent, let them vent before session begins
  • Be vigilant about observation being the majority of the session
  • Whenever possible, ensure real tasks are being demonstrated, from start to finish

Session Wrap-up

  • Thank participants for their time
  • Offer to answer any remaining questions that participants may have
  • Would be appropriate to now share any tips and/or ask for their own possible solutions
  • If appropriate, ask participants if follow-up contact is ok

Post Session Activities

  • Review collected data as soon as possible
  • Ensure all data/notes are traceable to individual participants
  • Perform task analysis on observed workflows (i.e. flowchart)
  • Consider construction of affinity diagram (a hierarchical organization of all notes)

Affinity Diagram

Most often a group activity involving all observers

Affinity Diagram

Individual notes/observations with participant # are written on
Post-it notes and grouped by common themes

Example Affinity Diagram

Copyright by respective copyright owners. Used without permission under the Fair Use Doctrine. Source: www.flickr.com

Contextual Inquiry Summary

  • What is a contextual inquiry?
  • Key principles
  • Possible formats
  • Inquiry plan and scripts
  • Probing for more information
  • Session tips
  • Session wrap-up
  • Affinity diagrams

Summary

  • Types of user research
  • Interviews
  • Contextual inquiries

References and Suggested Books

  • Contextual Design: Defining Customer-Centered Systems by Hugh Beyer and Karen Holtzblatt
  • Interviewing Users by Steve Portigal
  • Learning From Strangers: The Art and Method of
  • Qualitative Interview Studies by Robert Weiss
  • Observing the User Experience: A Practitioners Guide to User Research by Mike Kuniavsky
  • Rapid Contextual Design: A How-to Guide to Key Techniques for User-Centered Design by Karen Holtzblatt, Jessamyn Burns Wendell and Shelley Wood 
  • User and Task Analysis for Interface Design by JoAnn Hackos and Janice Redish

CMPT-363-153 User Research

By Paul Hibbitts

CMPT-363-153 User Research

  • 2,886