Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A developer's perspective: the problem with screen reader testing

jaketracey.com

Post reply on HN
Print view View original

41–50 of 54 posts

Re: A developer's perspective: the problem with screen reader testing

#41

Post by bramd »

Before doing screen reader testing on complex web components, what I see as some kind of lack box testing where you test your whole screen reader + browser stack, it is useful to have a look at what the browser passes to a screen reader. Especially Firefox has a very nice accessibility tree panel in the devtools these days. In my experience, the more visual tree that is shown there is also easier/faster to read for users that are not blind and are not that quick when using screen readers.

Also, keep in mind that something that technically works correctly with screen readers is just the beginning. User testing might reveal lots of issues you wouldn't think of yourself. And yes, I know that resources are usually limited and there is not much room for user testing, especially testing with screen reader users and other groups that have some kind of disability. I recently worked as the accessibility lead of a mobile COVID exposure notification app that had a very simple UI and a hard accessibility requirement. We had the luxury to do extensive user testing and even in this simple interface we found lots of small changes that improved the experience for screen reader users.

Re: A developer's perspective: the problem with screen reader testing

#42

Post by SpaceL10n »

epikur wrote: post #4
detaro wrote: post #2

In addition, I wish screenreaders had a text mode, where they print what they say and maybe provide cues on possible actions. Actual screenreader users work with astonishing speaking speeds and great memorization of commands, but without the experience a bare-bones but more visual interface would likely be easier to use.

They do! Not fully optimized for developers perhaps, but check out "Speech Viewer" in NVDA and the "Braille Viewer" and "Speech History" in JAWS. This guide has some screenshots: https://www.accessibility-developer-guide.com/setup/screen-r...

And, in addition to those JAWS tools, there is also JAWS Inspect (https://www.paciellogroup.com/products/jaws-inspect/)

Re: A developer's perspective: the problem with screen reader testing

#43

Post by fao_ »

bramd wrote: post #41

Before doing screen reader testing on complex web components, what I see as some kind of lack box testing where you test your whole screen reader + browser stack, it is useful to have a look at what the browser passes to a screen reader. Especially Firefox has a very nice accessibility tree panel in the devtools these days. In my experience, the more visual tree that is shown there is also easier/faster to read for u…

Would it be possible for you to do a writeup on what you found that might be transferrable?

Re: A developer's perspective: the problem with screen reader testing

#44

Post by mikkergp »

toast0 wrote: post #15
mwcampbell wrote: post #6

> Sadly, there’s not currently any way for a developer to identify the type or version of a screen reader that is being used That's at least partly because there are some vocal blind people who don't want websites to be able to know that they're running a screen reader at all, for fear of discrimination. I believe that stance is misguided. I'll illustrate why with a story. A few years ago, my best friend, who is blin…

I can't imagine websites doing anything but dropping screen reader users into a separate experience; rather than doing the work to design for accessibility in all their experiences and auditing and testing to confirm. Separate, but equal isn't equal.

I’m genuinely asking here out of lack of experience — Is it separate but equal, or is it customizing the user experience? I’m not blind but it always seemed weird to me to try and retrofit visual abstractions to someone who may not be as comfortable with those abstractions then to build an experience directly to that user. Mobile sites often have different experiences than desktop sites, and we would say that’s an experience customized for a small screen, not separate but equal. Wouldn’t designing a separate experience from the ground up built for someone with visual impairment be better than fitting them into a system that’s probably wasn’t set out for them in the first place? You’re hampering the experience because you’ll always be biasing for the sighted, unless your ux lead is visually impaired.

Re: A developer's perspective: the problem with screen reader testing

#45

Post by bramd »

fao_ wrote: post #43
bramd wrote: post #41

Before doing screen reader testing on complex web components, what I see as some kind of lack box testing where you test your whole screen reader + browser stack, it is useful to have a look at what the browser passes to a screen reader. Especially Firefox has a very nice accessibility tree panel in the devtools these days. In my experience, the more visual tree that is shown there is also easier/faster to read for u…

Would it be possible for you to do a writeup on what you found that might be transferrable?

Yes, I would like to publish some lessons in the future somewhere. However, a few quick takeaways:

* The microcopy matters, a lot. We had a button stating "I've got a notification: read what you should do after getting a notification" (from the top of my head and freely translated from Dutch, we didn't have an English translation back then). This was part of a bunch of buttons on the main screen that all gave information. Some screen reader users got confused and thought that they had a notification. If you don't see the visual layout, it is not obvious that this is just a plain button and not a bold text in red that is giving you a warning. * In the same category: the app has a status text that says "The app is working fine" or "The app is not working fine". Visually, the error state is signified by an exclamation mark and styling that makes clear that this is a serious issue. However, in text there is just one word, not, to signify that there is a serious issue. Following WCAG, the info signified by the exclamation mark icon was available in text, so no text alternative was required. However, we gave it a text alternative anyway to ensure screen reader users were also clearly alerted that something is wrong. Same goes for the "all is ok" icon, we gave that one a text alternative as well to ensure users all is fine.

Re: A developer's perspective: the problem with screen reader testing

#46

Post by jscholes »

mikkergp wrote: post #44
toast0 wrote: post #15

Earlier quoted context omitted.

I can't imagine websites doing anything but dropping screen reader users into a separate experience; rather than doing the work to design for accessibility in all their experiences and auditing and testing to confirm. Separate, but equal isn't equal.

I’m genuinely asking here out of lack of experience — Is it separate but equal, or is it customizing the user experience? I’m not blind but it always seemed weird to me to try and retrofit visual abstractions to someone who may not be as comfortable with those abstractions then to build an experience directly to that user. Mobile sites often have different experiences than desktop sites, and we would say that’s an ex…

> I’m genuinely asking here out of lack of experience — Is it separate but equal, or is it customizing the user experience?

It's separation, because it introduces the chances of an inequitable experience being created as the "customised" version for screen reader users drifts out of sync. This may not even be down to anything malicious; plenty of companies create mobile-friendly versions of websites and apps, only to find that as years go by, they no longer have the budget to facilitate their upkeep. Far from assigning them to the trash heap of history, these often continue to be provided, offering a substandard experience, and this is despite companies having so much data available about high mobile usage. The chances of it happening to an accessibility-specific version are higher because the likely user take-up is lower, and as this post and thread indicate, speciality skills are often required to make a good job of it. Not like responsive web design, where experts are ten a penny.

There are other reasons this approach is flawed, of course:

1. Screen reader users aren't the only disabled people out there. Heck, even within that group, there are those who use the software as an absolute necessity, and others who use it in addition with other assistive tech like speech recognition or a magnifier. Nobody in their right mind is going to create a separate experience for every subset of disabled users. Even if they did, how would they be surfaced?

2. Some people are only temporarily disabled, e.g. because of an injury. They don't have the lifelong feel for how to look for accessibility settings or separate modes, so they aren't likely to benefit from a shiny accessible version. Making your main product inclusive prepares for this.

3. The aim should be to allow customisations to be included which aren't intrusive to those who don't need them, e.g. via techniques like WAI-ARIA[1] which allow additional context to be added for screen reader users while remaining invisible to everyone else.

[1] https://www.w3.org/TR/wai-aria-1.2/

Re: A developer's perspective: the problem with screen reader testing

#47

Post by jscholes »

bryanrasmussen wrote: post #37
toast0 wrote: post #15

Earlier quoted context omitted.

I can't imagine websites doing anything but dropping screen reader users into a separate experience; rather than doing the work to design for accessibility in all their experiences and auditing and testing to confirm. Separate, but equal isn't equal.

well, the fact is the use of a screen reader (or at least most) can be detected relatively easily by having an element in the screen reader flow that is hidden from sighted users, and if you wanted for extra surety having an element that will be interacted with by anyone accessing the page - but that is hidden from screen readers but available to sighted users. detecting interaction with visually hidden element, but…

> well, the fact is the use of a screen reader (or at least most) can be detected relatively easily by having an element in the screen reader flow that is hidden from sighted users

This isn't true, at least not enough to be useful. The majority of screen readers present web pages in a kind of virtual buffer, which users can browse with the arrow keys and other shortcuts (e.g. H to jump by heading). In this mode, the majority of screen readers don't fire focus events to let you know, "hey, your screen-reader-only element has been hit". You would be limited to:

1. users operating a screen reader which does fire focus events;

2. trying to highjack scroll events for this purpose; and

3. less technically-inclined users (read: beginners) who move through the page with Tab/Shift+Tab before they've learned the other keystrokes their screen reader offers.

Re: A developer's perspective: the problem with screen reader testing

#48

Post by jscholes »

Vinnl wrote: post #39

There are a number of browsers that run in Docker and that I can remote control (using Selenium, Playwright, Puppeteer or whatever) in my CI systems to run at least some smoke tests, making sure that basic features are available. Does anyone know if something like that is available for screen readers - at least for a free and open source one?

> There are a number of browsers that run in Docker and that I can remote control (using Selenium, Playwright, Puppeteer or whatever) ... > > Does anyone know if something like that is available for screen readers - at least for a free and open source one?

Nothing in this space is really mature yet, but there are some efforts to make it a reality. The ARIA-AT project[1], which aims to test assistive technology support for various WAI-ARIA and HTML constructs, is aiming to automate its testing across multiple screen readers [2]. NVDA, the free and open source screen reader mentioned in the article, also includes some integration-style tests[3].

[1] https://github.com/w3c/aria-at [2] https://github.com/w3c/aria-at/issues/349 [3] https://github.com/nvaccess/nvda/tree/master/tests/system

Re: A developer's perspective: the problem with screen reader testing

#49

Post by jscholes »

vagab0nd wrote: post #40

Is there some software library that turns websites into text, that these screen readers all use, or do they implement their own?

> Is there some software library that turns websites into text, that these screen readers all use, or do they implement their own?

They implement their own. But the browser also has responsibilities in this area, to construct an accessibility tree from the DOM which screen readers can parse.

Re: A developer's perspective: the problem with screen reader testing

#50

Post by Vinnl »

jscholes wrote: post #48
Vinnl wrote: post #39

There are a number of browsers that run in Docker and that I can remote control (using Selenium, Playwright, Puppeteer or whatever) in my CI systems to run at least some smoke tests, making sure that basic features are available. Does anyone know if something like that is available for screen readers - at least for a free and open source one?

> There are a number of browsers that run in Docker and that I can remote control (using Selenium, Playwright, Puppeteer or whatever) ... > > Does anyone know if something like that is available for screen readers - at least for a free and open source one? Nothing in this space is really mature yet, but there are some efforts to make it a reality. The ARIA-AT project[1], which aims to test assistive technology suppor…

Great to hear that people are working on this, thanks for sharing. Looking forward to seeing that work mature.
Post reply on HN