Pulse oximeters give biased results for people with darker skin • Hacker News Board

Re: Pulse oximeters give biased results for people with darker skin

#41

Post by ravenstine » Thu, Aug 06, 2020, 3:19 PM UTC

exikyut wrote: post #36

ravenstine wrote: post #15

Earlier quoted context omitted.

I can believe this because automatic doors seem more prone to having a hard time opening when I wear black clothes. It's not a big deal or anything, but I occasionally have to step back and then forth again to get it to open. Is this a common experience of black people, or am I crazy?

Oh, THAT'S why entrance barriers at supermarkets only occasionally fail, but fail consistently on a given day... Wait. Would this happen to PIR-based systems?

I'm not sure. It seems at least possible with supermarket doors because those supposedly use a low-power microwave beam. With PIR, it still relies on a deflection of a beam, so I think it would be a problem with that as well, but I'm just guessing.

Re: Pulse oximeters give biased results for people with darker skin

#42

Post by tolbish » Thu, Aug 06, 2020, 3:22 PM UTC

oneplane wrote: post #22

And darker coloured textiles are less visible on people with darker skin. Grass is green and bananas tend to be curved. I get that it is interesting to research optically measuring systems to 'search for bias' and I'm sure there are going to be biases in varying degrees, but that doesn't mean it's always for the same reason or with the same result. There are very real problems of course, take the example with the pho…

Just because your customer didn't explicitly tell you to account for people with darker skin doesn't mean your failure to do so is just "an error made during engineering". It can still be systemic racial bias for neither you nor your client to consider who is actually going to be using your devices.

Also, no one said this is the only problem we have with devices.

Re: Pulse oximeters give biased results for people with darker skin

#43

Post by krapp » Thu, Aug 06, 2020, 3:22 PM UTC

jeffreyrogers wrote: post #29

And people with sweaty hands. And people with low iron levels. There's this push to show that everything is biased against certain classes. I'm sure some of the examples are true, but I can't imagine that constantly seeking evidence of problematic issues is good for individuals or society.

Seeking evidence of problematic issues is how society identifies and eventually addresses such issues. Do you believe society would benefit from ignoring evidence or pretending those issues didn't exist?

Re: Pulse oximeters give biased results for people with darker skin

#44

Post by vmception » Thu, Aug 06, 2020, 3:22 PM UTC

oneplane wrote: post #22

And darker coloured textiles are less visible on people with darker skin. Grass is green and bananas tend to be curved. I get that it is interesting to research optically measuring systems to 'search for bias' and I'm sure there are going to be biases in varying degrees, but that doesn't mean it's always for the same reason or with the same result. There are very real problems of course, take the example with the pho…

The product just wouldn't have shipped with a product manager who also experienced it not working.

Thats the point of inclusion.

Although the soap dispenser example feels like one of those problems, it is just wear and tear of the sensor and they work properly when cleaned.

In this case, pulseox failures, can have effects on when people are admitted for treatment.

Re: Pulse oximeters give biased results for people with darker skin

#45

Post by jeffreyrogers » Thu, Aug 06, 2020, 3:23 PM UTC

x86_64Ubuntu wrote: post #37

jeffreyrogers wrote: post #29

And people with sweaty hands. And people with low iron levels. There's this push to show that everything is biased against certain classes. I'm sure some of the examples are true, but I can't imagine that constantly seeking evidence of problematic issues is good for individuals or society.

It sounds like you are suffering from anxiety over the phrase "darker skin".

Since we're psycho-analyzing each other via HN comments, I think you're projecting.

Re: Pulse oximeters give biased results for people with darker skin

#46

Post by jeffreyrogers » Thu, Aug 06, 2020, 3:25 PM UTC

krapp wrote: post #43

jeffreyrogers wrote: post #29

And people with sweaty hands. And people with low iron levels. There's this push to show that everything is biased against certain classes. I'm sure some of the examples are true, but I can't imagine that constantly seeking evidence of problematic issues is good for individuals or society.

Seeking evidence of problematic issues is how society identifies and eventually addresses such issues. Do you believe society would benefit from ignoring evidence or pretending those issues didn't exist?

I agree that seeking evidence is how issues are addressed. The problem is that evidence is being sought in a particular way, which systematically biases which evidence is available for consideration.

Edit: there is a further problem that because many interpretations of the same facts are possible if you go looking for certain things you will very likely find them. While someone with different initial beliefs will find evidence of something else. All from the same facts.

Re: Pulse oximeters give biased results for people with darker skin

#47

Post by jfengel » Thu, Aug 06, 2020, 3:26 PM UTC

oneplane wrote: post #22

And darker coloured textiles are less visible on people with darker skin. Grass is green and bananas tend to be curved. I get that it is interesting to research optically measuring systems to 'search for bias' and I'm sure there are going to be biases in varying degrees, but that doesn't mean it's always for the same reason or with the same result. There are very real problems of course, take the example with the pho…

Each instance of it isn't necessarily malicious, but the way they all end up causing harm to the same people makes the system as a whole act as if it were malicious. Especially when combined with a proportion of people who are actively malicious, and when any attempts to address the systemic issue are dismissed.

When it comes to us in particular, we can each truthfully say "it wasn't my fault and I didn't mean any harm". But harm happens, and Hanlon's razor becomes dull with repeated application like that. As another rule goes, "Once is happenstance, twice is coincidence, three times is enemy action".

The solutions to systemic problems aren't simple. They certainly include rejecting the people who are outright being malicious. They also include a willingness to say, "Gosh, I keep using this didn't-mean-anything excuse. Maybe I need to start instead meaning to not do it."

Re: Pulse oximeters give biased results for people with darker skin

#48

Post by oneplane » Thu, Aug 06, 2020, 3:31 PM UTC

vmception wrote: post #44

oneplane wrote: post #22

And darker coloured textiles are less visible on people with darker skin. Grass is green and bananas tend to be curved. I get that it is interesting to research optically measuring systems to 'search for bias' and I'm sure there are going to be biases in varying degrees, but that doesn't mean it's always for the same reason or with the same result. There are very real problems of course, take the example with the pho…

The product just wouldn't have shipped with a product manager who also experienced it not working. Thats the point of inclusion. Although the soap dispenser example feels like one of those problems, it is just wear and tear of the sensor and they work properly when cleaned. In this case, pulseox failures, can have effects on when people are admitted for treatment.

It would probably not have shipped in that case, but the amount of product managers actually using their own products sadly isn't at the saturation you'd expect.

Same with all the white labeling and supply chain and integrators in between; the disconnect between all of the stages and the contracts that can bind a bad product to be delivered and sold anyway can't be fixed on just one end of the process.

Re: Pulse oximeters give biased results for people with darker skin

#49

Post by vinay427 » Thu, Aug 06, 2020, 3:34 PM UTC

ceejayoz wrote: post #23

vinay427 wrote: post #20

Earlier quoted context omitted.

The Apple Watch doesn't have a pulse oximeter (EDIT: it does have one that is not approved and therefore is disabled), so it would be difficult for it to have this issue. You might be thinking of optical HR measurement, for which sensors have also been shown to have skin color biases in some cases. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7010823/

> The Apple Watch doesn't have a pulse oximeter Yes, it does. The FDA blocked its use. https://www.city-journal.org/fda-blocks-apple-watch-blood-ox... https://www.cultofmac.com/320322/apple-watch-sensors-are-cap...

Noted and thanks for the information, but as far as I can tell my point stands if the feature is completely inaccessible. I don't really understand what the GP comment was referring to regarding Apple Watch pulse oximetry accuracy. Other consumer devices such as my smartphone and watch have working pulse oximeter sensors built-in that are presumably FDA approved (or not FDA banned anyway), so it doesn't bode well for the Apple Watch if it can't pass this.

Re: Pulse oximeters give biased results for people with darker skin

#50

Post by oneplane » Thu, Aug 06, 2020, 3:37 PM UTC

jfengel wrote: post #47

oneplane wrote: post #22

And darker coloured textiles are less visible on people with darker skin. Grass is green and bananas tend to be curved. I get that it is interesting to research optically measuring systems to 'search for bias' and I'm sure there are going to be biases in varying degrees, but that doesn't mean it's always for the same reason or with the same result. There are very real problems of course, take the example with the pho…

Each instance of it isn't necessarily malicious, but the way they all end up causing harm to the same people makes the system as a whole act as if it were malicious. Especially when combined with a proportion of people who are actively malicious, and when any attempts to address the systemic issue are dismissed. When it comes to us in particular, we can each truthfully say "it wasn't my fault and I didn't mean any ha…

The issue with all of this, is that it's not just one company with a group of all the parts that design, create, test and sell this stuff.

Most of it was designed somewhere, built somewhere else, integrated yet somewhere else, and then simply bought out of a catalogue bulk, shipped to a warehouse and sold in smaller bulk to stores. The products and people that could have detected a problem are all over the world, and have seen every possible version of a human and still they end up in use with the problems they have.

If the pattern were constrained to fit a scenario as you describe then it would probably be something we can do something about. But just like in enterprise software contracts, the distance (both mentally, physically and legally) between all of this is not allowing for much change for any single actor. And I'm pretty sure the manufacturer of a sensor that doesn't even know where it ends up 10 layers deep in the supply chain is going to be remotely aware of this.

In practical terms we're seeing an anti-pattern and we try to personify it so we can express our dissatisfaction, but that's just as practical as shouting at the sun for not damaging all skin types at similar rates. There is no single source of the fault and there is no person or company or something like that, that is 'responsible' for it.

One possibility is to have some sort of consumer organisation test products on a diverse set of humans and report on products that lack support for certain shapes, colours or sizes, but other than that I have no idea how we'd adjust multiple people at multiple companies in multiple countries with multiple cultures.