Hands-on Accessibility Testing

Eric Eggert · European Testing Conference ·

Eric Eggert

Web Accessibility Specialist @

Knowbility (Assessments, Teaching)

50% W3C Fellow → WAI/EOWG

Prepare!

Install three Bookmarklets:

  1. Go to: l.knowbility.org/bml
  2. Drag the following bookmarklets to your favorites toolbar:
    • ​Images
    • Headings
    • Force Show Focus

What’s Accessibility?

“The inclusive practice of making websites¹ usable by everyone,
regardless of ability or disability.”

¹And Apps!

Affected People

  • People with Disabilities
  • People with temporary impairments
  • People with other functional limitations

No way to detect, no way to count

Beyoncé sued over website accessibility A class action suit claims that Beyonce.com does not cater to visually impaired users.

The Business Case

for Digital Accessibility

Accessible design is by its nature flexible, allowing content to faithfully render across a broad spectrum of devices, platforms, assistive technologies, and operating systems.

In physical environments, everyone takes advantage of lower curbs, automatic door openers, ramps, and other features provided for disability access.

On the web, accessibility features become options that are also often used more widely.”

15–20 % of people have a disability

👵🏻👵🏿 👵🏼👴🏾 👴🏼👵🏽 👵🏾👴🏿 👴🏻👴🏽

Age & Disability

Age group Prevalence
21–64 10.5 %
65–74 25.6 %
75+ 50.3 %

Design a Web
For Future You

All Statistics from: http://disabilitystatistics.org

What are we testing against?

Most accessibility problems are common across sites/apps.

WCAG 2:

Web Content Accessibility Guidelines

Four Principles

  • 1. Perceivable
  • 2. Operable
  • 3. Understandable
  • 4. Robust

13 Guidelines

  • 1.1 Text Alternativest
  • 1.2 Time-based Media
  • 1.3 Adaptable
  • 1.4 Distinguishable
  • 2.1 Keyboard Accessible
  • 2.2 Enough Time
  • 2.3 Seizures and Physical Reactions
  • 2.4 Navigable
  • 2.5 Input Modalities
  • 3.1 Readable
  • 3.2 Predictable
  • 3.3 Input Assistance
  • 4.1 Compatible

13 Guidelines

  • 1.1 Text Alternativest
  • 1.2 Time-based Media
  • 1.3 Adaptable
  • 1.4 Distinguishable
  • 2.1 Keyboard Accessible
  • 2.2 Enough Time
  • 2.3 Seizures and Physical Reactions
  • 2.4 Navigable
  • 2.5 Input Modalities
  • 3.1 Readable
  • 3.2 Predictable
  • 3.3 Input Assistance
  • 4.1 Compatible

78 Success Criteria

  • 1.1.1 Non-text Content
  • 1.2.1 Audio-only and Video-only (Prerecorded)
  • 1.2.2 Captions (Prerecorded)
  • 1.2.3 Audio Description or Media Alternative (Prerecorded)
  • 1.2.4 Captions (Live)
  • 1.2.5 Audio Description (Prerecorded)
  • 1.2.6 Sign Language (Prerecorded)
  • 1.2.7 Extended Audio Description (Prerecorded)
  • 1.2.8 Media Alternative (Prerecorded)
  • 1.2.9 Audio-only (Live)
  • 1.3.1 Info and Relationships
  • 1.3.2 Meaningful Sequence
  • 1.3.3 Sensory Characteristics
  • 1.3.4 Orientation
  • 1.3.5 Identify Input Purpose
  • 1.3.6 Identify Purpose
  • 1.4.1 Use of Color
  • 1.4.2 Audio Control
  • 1.4.3 Contrast (Minimum)
  • 1.4.4 Resize text
  • 1.4.5 Images of Text
  • 1.4.6 Contrast (Enhanced)
  • 1.4.7 Low or No Background Audio
  • 1.4.8 Visual Presentation
  • 1.4.9 Images of Text (No Exception)
  • 1.4.10  Reflow
  • 1.4.11  Non-text Contrast

78 Success Criteria

  • 1.4.12  Text Spacing
  • 1.4.13  Content on Hover or Focus
  • 2.1.1 Keyboard
  • 2.1.2 No Keyboard Trap
  • 2.1.3 Keyboard (No Exception)
  • 2.1.4 Character Key Shortcuts
  • 2.2.1 Timing Adjustable
  • 2.2.2 Pause, Stop, Hide
  • 2.2.3 No Timing
  • 2.2.4 Interruptions
  • 2.2.5 Re-authenticating
  • 2.2.6 Timeouts
  • 2.3.1 Three Flashes or Below Threshold
  • 2.3.2 Three Flashes
  • 2.3.3 Animation from Interactions
  • 2.4.1 Bypass Blocks
  • 2.4.2 Page Titled
  • 2.4.3 Focus Order
  • 2.4.4 Link Purpose (In Context)
  • 2.4.5 Multiple Ways
  • 2.4.6 Headings and Labels
  • 2.4.7 Focus Visible
  • 2.4.8 Location
  • 2.4.9 Link Purpose (Link Only)
  • 2.4.10  Section Headings
  • 2.5.1 Pointer Gestures
  • 2.5.2 Pointer Cancellation
  • 2.5.3 Label in Name
  • 2.5.4 Motion Actuation

78 Success Criteria

  • 2.5.5 Target Size
  • 2.5.6 Concurrent Input Mechanisms
  • 3.1.1 Language of Page
  • 3.1.2 Language of Parts
  • 3.1.3 Unusual Words
  • 3.1.4 Abbreviations
  • 3.1.5 Reading Level
  • 3.1.6 Pronunciation
  • 3.2.1 On Focus
  • 3.2.2 On Input
  • 3.2.3 Consistent Navigation
  • 3.2.4 Consistent Identification
  • 3.2.5 Change on Request
  • 3.3.1 Error Identification
  • 3.3.2 Labels or Instructions
  • 3.3.3 Error Suggestion
  • 3.3.4 Error Prevention (Legal, Financial, Data)
  • 3.3.5 Help
  • 3.3.6 Error Prevention (All)
  • 4.1.1 Parsing
  • 4.1.2 Name, Role, Value
  • 4.1.3 Status Messages​

WCAG-EM:

Website Accessibility Conformance Evaluation Methodology

  • Step 1: Define the Evaluation Scope
  • Step 2: Explore the Target Website
  • Step 3: Select a Representative Sample
  • Step 4: Audit the Selected Sample
  • Step 5: Report the Evaluation Findings

Tooling

Testability of Best Practices by WCAG* Level

WCAG Level Automatic Test Manual Verification Manual Test
A 25% 29% 46%
AA 17% 41% 41%
AAA 23% 24% 53%

*WCAG 2.0, Source: http://www.karlgroves.com/2012/09/15/accessibility-testing-what-can-be-tested-and-how/

Online Tools

Most tools run everywhere!

However there might be some that are specific to a browser or operating system.

Firefox
Browser Plugins & Toolbars

The Problems with such tools?

Testing with Screen Readers

Jaws
NV Access NVDA

VoiceOver

CTRL+OPTIONS = VO

http://bit.ly/Webaim-VO

The Problems with SR testing by People without a Disability?

Integrate ALL THE THINGS
But HOW?

Command Line Tools

Introduce Accessibility as

a Requirement from the Start

Expert Reviews

User Acceptance Testing

User Acceptance Testing
is often reduced to UX testing,
is often reduced to UI testing,
is often reduced to guessing,
is often reduced to nothing.

User Acceptance Testing

is not testing the user.

But how they are using your product.

Have your website tested by users. (Remotely.)

  • Unmoderated testing
  • Find challenges on your website for people with specific disabilities
    • Five major categories of registered testers: vision, hearing, motor, cognitive, and neurological
  • Fast, useful feedback
  • For every size of organization
  • Full privacy for the testers
  • In partnership with UserZoom and Loop11

If you can, try to hire PwD.

The contact and insight of real-life accessibility are worth far more than any experts efforts.

The Future of Accessibility Testing

Accessibility Conformance Testing Task Force at W3C

Thank You

Eric Eggert

Knowbility, yatil.net, @yatil

 

https://slides.com/yatil/2019-etc-a11y-testing

Hands-on Accessibility Testing

By Eric Eggert

Hands-on Accessibility Testing

  • 2,467