Accessibility testing without real users is not enough

Testing tools can only tell you what is detectable. What they can't reveal is whether an experience that technically works is actually easy to use. And that's where the gap sits.

The numbers are worth knowing. Automated tools reliably catch somewhere under half of the issues defined in WCAGglossary×WCAGWCAG is a set of guidelines for making digital content accessible to people with disabilities.Open glossary term→ itself, and the ones they catch are largely the mechanical ones: missing alternative text, contrast, unlabelled inputs. Everything requiring a judgement about whether something makes sense is outside what a scanner can assess.

A clean report is therefore a statement about a subset. It is genuinely useful, and it is routinely read as a conclusion about the whole product rather than about the part a machine was able to check.

You can have a product that performs well in automated tests and still creates significant frictionglossary×FrictionFriction refers to anything that slows users down or makes it harder for them to complete a task. It can be caused by poor design, unnecessary steps, unclear messaging, or technical issues.Open glossary term→ for real users. Journeys can technically work but require unnecessary effort. Content can be accessible but difficult to understand in contextglossary×ContextContext is the set of surrounding conditions that shape how someone behaves and decides, including their goal, their environment, their time pressure, and what happened immediately before. The same action can mean different things in different contexts.Open glossary term→. Interactionsglossary×InteractionInteraction refers to any action a user takes within a product and how the system responds. It includes clicks, taps, gestures, and inputs that drive the user experience.Open glossary term→ can be valid but unintuitive when used differently than originally intended. Nothing is technically wrong, but something is clearly not working. That only becomes visible when real people start using it.

The gap is easiest to see in timings. Two people complete the same task and one takes four times as long, with no error, no failure, and nothing an audit would flag. The product works for both of them. It is only usable for one.

Users who rely on assistive technology don't interact with products the same way as those designing or building them. Screen readerglossary×Screen ReaderA screen reader is software that reads digital content aloud for users who cannot see the screen.Open glossary term→ users navigate differently. Keyboard-only users move through content in a different rhythm. Voice control introduces another layer of interactionglossary×InteractionInteraction refers to any action a user takes within a product and how the system responds. It includes clicks, taps, gestures, and inputs that drive the user experience.Open glossary term→ entirely. These aren't edge casesglossary×Edge CaseAn edge case is a rare or extreme scenario that falls outside typical user behaviour.Open glossary term→. They're real usage patterns. And they expose issues that tools cannot.

A screen readerglossary×Screen ReaderA screen reader is software that reads digital content aloud for users who cannot see the screen.Open glossary term→ might be able to access every element on a page, but the order in which those elements are read can make the journey confusing. A form might be fully operable via keyboard but require too many steps to complete efficiently. Navigationglossary×NavigationNavigation is the system of menus, links, and cues that lets people move through a product and reach what they need. It exposes the underlying structure, and is how most users experience that structure.Open glossary term→ might technically work but lack the contextglossary×ContextContext is the set of surrounding conditions that shape how someone behaves and decides, including their goal, their environment, their time pressure, and what happened immediately before. The same action can mean different things in different contexts.Open glossary term→ needed to make decisions quickly. From a compliance perspective, everything passes. From a user perspective, the experience breaks down.

Expertise cuts the other way too, and it catches teams out. Experienced screen readerglossary×Screen ReaderA screen reader is software that reads digital content aloud for users who cannot see the screen.Open glossary term→ users are fast, and will get through journeys that defeat somebody less practised. Testing only with confident users produces a reassuring result about a product that still fails the people encountering it for the first time.

Teams often underestimate how early this kind of testing should happen. It's frequently left until later stages (once the product is more complete) under the assumption that it's something to validate rather than something to shape the design. By then, insightsglossary×InsightAn insight is a meaningful understanding that explains why something is happening and what it means.Open glossary term→ become harder to act on. Changes require more effort, timelines are tighter, and there's less flexibility to rethink core decisions.

When real user testing is brought in earlier, it becomes part of how decisions are made, not just how they are checked. Assumptions are challenged sooner. Patternsglossary×PatternA pattern is a reusable solution to a design problem that recurs across products and contexts. It captures an approach already shown to work, so a team solves the problem once rather than every time it appears.Open glossary term→ are refined before they are scaled. Journeys are shaped based on actual behaviourglossary×BehaviourBehaviour refers to how users interact with a system, including actions, patterns, and responses.Open glossary term→ rather than being adjusted after the fact.

Automated tools and audits still have a place. They provide consistencyglossary×ConsistencyConsistency is the use of uniform patterns, behaviours, and visual elements across a product to create familiarity and predictability. It helps users learn once and apply that knowledge throughout the experience.Open glossary term→, speed, and coverage that would be difficult to achieve otherwise. But they shouldn't be the only source of truth. Without real user input, you're only seeing part of the picture. Accessibilityservice×AccessibilityFind accessibility issues early, improve usability, and build products that are more inclusive, usable, and compliant.Open service→ isn't proven by passing tests. It's proven by people being able to use the product without unnecessary effort.

The cheapest versionglossary×VersionA version is a specific iteration of software or a product at a point in time.Open glossary term→ of earlier is not a study at all. It is including one or two people who use assistive technology in the usabilityglossary×UsabilityUsability is how easy and efficient it is for users to complete tasks within a product. It focuses on clarity, simplicity, and reducing effort so users can achieve their goals without confusion or friction.Open glossary term→ sessionsglossary×SessionA session is a single period of user interaction with a product, from entry to exit within a defined timeframe.Open glossary term→ you are already running, rather than treating accessibility as a separate activity with its own budget, timeline and approval.