Inclusive user testing: a quick guide

Florence Okoye

@FINOkoye

It can seem complicated...

...but like a lot of things, usability testing can be fun when you break it down.


By the end of this presentation you will hopefully feel a bit more confident about how to do usability testing.

 

@FINOkoye

Think of testing as another part of discovery...

“Design isn’t finished until somebody is using it.”

Brenda Laurel

@FINOkoye

1. What am I testing?

  • What are the product goals? 
    • This impacts what gets tested and how
  • Who needs to know? 
    • Do I need a full report, a quick presentation, a check-in with technicians?
  • What is complexity of the product? 
    • This impacts how many people you should include.

When you first start the design process you might have created a product statement, or have a set of KPIs to meet.

@FINOkoye

Are users achieving their goals?

  • This often means getting more open, qualitative feedback

But how do people rate their experience? Success from a business perspective isn’t always the same from a user perspective and this is where gaps and scope creep can develop. The two must align to say a product has good user experience.


People are good at finding workarounds so it’s important to also judge by their own standards. You might get interesting contrasts when compared to usability, ease of use and business metrics!

Are users achieving business goals?

  • It usually helps to get quantitative measures e.g. task completion rate

 

@FINOkoye

2. Who am I testing with?

As part of the initial research, you will probably have:

  • Personas (or similar)
  • Scenarios
  • User journeys

 

Personas are useful for understanding who to include in the user testing.


However, don’t just recruit people who fit the profile. Non-target audiences can tell you how intuitive a product is.

@FINOkoye

Personas generally don’t include demographic or marginalised characteristics (unless that is specifically what the product is for), so you need to make sure the pool of testers is as diverse as possible.

 

Join forums, use social media,  etc. ask specific charities for help.

 

There are many ways to do this, it often just requires a bit of imagination and google search!

@FINOkoye

The way we treat people is also key to human centred design.

Before testing, make sure you provide a consent form that explains:

  • What the test involves e.g. interview, being recorded etc.
  • How the data will be gathered, stored and used, and by who
  • Who the person can contact if they want to withdraw their data

 

@FINOkoye

3. When am I testing?

We often say test early and often but in order to be impactful, this means testing when you can still do the changes.

This means agreeing with the rest of the team about when testing can be done, what the outputs should be and what should be done with the results (within what timeframe).


Involve everyone as much as possible. Your teammates can help out with everything from giving feedback on questions to watching recordings.

@FINOkoye

4. How am I testing?

For example, if you want to find out what people think about an interface/concept you might want to do a  short impression test or word association activity before you ask them to do any tasks.

There are different ways to test a product depending on what you want to learn.


Understanding how bias can come into the results will also help you structure the testing and use the appropriate tools.

 

@FINOkoye

There are many tools you can use for remote testing.

 

However as a general guidance, the more complex a user journey, or the more dependencies a scenario includes, or the more unknowns there are, the greater the need for in person testing, so all the intricacies can be captured.

@FINOkoye

Open questions are always best but sometimes you will need to dig into particular features.


Structure the test to do the feature specific feedback after the open task, to avoid familiarity bias.

Using the set of scenarios you created during the research process, you can simply ask people how they would do a particular thing using your product.


Using rating questions like the SEQ (Single Ease Question) to get a quantitative baseline for the experience. This is also great to compare against their actual success of following a proposed user journey.

 

@FINOkoye

Just as there are different ways of testing, there are many different tools you can use to test. It’s rare to find one tool that fits everything!

Some useful free tools for doing user testing:

 

  • Hotjar (for recruiting from a website) 
  • Google forms (for surveys with automated data visualisations) 
  • Jotform for surveys
  • Usability hub (free for short tests) 
  • jitsi for hosting moderated remote sessions
  • Signal for remote observation/contextual feedback

@FINOkoye

But before you start testing you'll need a...

 

  • A script - this should be tested in advance to make sure the flow is right and the questions make sense.
  • A spreadsheet - make sure you’ve set up where all your notes and scores can be captured for easy comparison

@FINOkoye

Create a testing plan for your prototype

Looking at the prototype and the user journey: How will you know if it works - what might you be able to measure? What won’t you be able to measure?

@FINOkoye

So, you can ensure you are starting well if you have...

  • Consent form
  • Budget
  • Diverse participants
  • A validated script
  • A well structured spreadsheet 
  • Communication plan (to the team/stakeholders)

@FINOkoye

Thank you for reading!