Live data from Hacker News

Pulse oximeters give biased results for people with darker skin

bostonreview.net

21–30 of 124 posts

Re: Pulse oximeters give biased results for people with darker skin

#21
I'm a student EMT and our instructor told us not to rely on a pulse ox reading if our patient has dark skin, or if they were in a fire, or are a lifelong smoker, or if they have any nail polish or dirt/grease on their fingertips. We just look for the 94-100% range and would only take action if we get a reading around 87% or lower, or see a sudden drop during treatment.

There are a few alternatives designs that aim to mitigate this particular issue by either preventing crosstalk [1] or by using multiple LED wavelengths and angles - the second one is something a med device engineer told me about a few months ago, but I'm having trouble finding any papers or information on it.

[1] https://www.hindawi.com/journals/jhe/2018/3521738/

Re: Pulse oximeters give biased results for people with darker skin

#22
And darker coloured textiles are less visible on people with darker skin. Grass is green and bananas tend to be curved.

I get that it is interesting to research optically measuring systems to 'search for bias' and I'm sure there are going to be biases in varying degrees, but that doesn't mean it's always for the same reason or with the same result.

There are very real problems of course, take the example with the photographic film development from the article: that is a problem because a picture that is supposed to reflect the subject you photographed no longer does what it was suppsoed to do. And someone here posted the example of the soap dispenser that couldn't "see" people that didn't have light skin. Also a problem, because now you can't get soap. Both of those are mostly examples of errors made during engineering (apply Hanlon's razor when in doubt - it's not always malice like people often assume), and might be preventable if there were more different samples and interactions available during development.

That doesn't mean it's the only type of problem we have with devices for humans, or that keeping a color palette of people around is always going to fix it.

Take the soap dispenser example: you could have made one that uses radar or ultrasonic detection, solves multiple problems because you now don't have to consider lighting conditions which always vary due to differences in bathrooms (light fixtures, positioning, positioning of the dispenser itself etc). That doesn't relate to skin color, but can just as easily be a similar problem that could have been prevented all the same by doing proper engineering. But it's likely that in cases like this (soap dispensers) the detection system was just a module that some other third party company imports and sells, and they bought it from a module integrator somewhere else and they bought the parts as generic motion detectors from a company 4 layers down the chain that specializes in measuring daytime movements of rabbits or something silly like that.

On the other hand: if you have a government that requires certain controls to be in place and applied for medical equipment to be used in a medical setting you would expect some of the requirements to be drawn up in a way that reflects the use cases. It appears that either the devices are not scoped within those controls, or the controls failed. Or it's all good and the difference in measurement is not significant enough to matter for the use case and thus the devices are used as-is and this research doesn't relate to the real world as well as people think.

Re: Pulse oximeters give biased results for people with darker skin

#23
post #2

Quelle surprise - garbage in, garbage out. I'll bet my Apple Watch doesn't have these issues.

The Apple Watch doesn't have a pulse oximeter (EDIT: it does have one that is not approved and therefore is disabled), so it would be difficult for it to have this issue. You might be thinking of optical HR measurement, for which sensors have also been shown to have skin color biases in some cases. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7010823/

> The Apple Watch doesn't have a pulse oximeter

Yes, it does. The FDA blocked its use.

https://www.city-journal.org/fda-blocks-apple-watch-blood-ox...

https://www.cultofmac.com/320322/apple-watch-sensors-are-cap...

Re: Pulse oximeters give biased results for people with darker skin

#25

I'm a student EMT and our instructor told us not to rely on a pulse ox reading if our patient has dark skin, or if they were in a fire, or are a lifelong smoker, or if they have any nail polish or dirt/grease on their fingertips. We just look for the 94-100% range and would only take action if we get a reading around 87% or lower, or see a sudden drop during treatment. There are a few alternatives designs that aim to…

The article notes that the devices inaccurately report higher oxygen levels for darker skin tones. Meaning if someone dark skinned reports a result that falls into the 94-100% range, they may in fact be at lower oxygen levels that put them at risk.

Re: Pulse oximeters give biased results for people with darker skin

#26
post #13

This really isn't a big deal. Doctors know that these pulse ox meters aren't very accurate and use them accordingly. Nobody's changing their plan because you have a pulse ox of 93 instead of 95. They're there to quickly measure big changes that indicate a problem. In other words, they're looking for patients going from 98 to 75, not 98 to 95.

What's the point of casually dismissing this article's premise? Whether the conclusions are fully sound or not isn't the goal of this article, this is journalistic not academic. The goal is to raise awareness of a potential bias, and I think it achieves that (assuming the facts all check out).

For one, according to the article the discrepancy was found to be up to 8 points in some cases, not 3. And it seems reasonable that quick, practical decisions in hospitals are made based on point thresholds, not purely huge point leaps as you suggest.

I haven't dug any further than the article and your comment to verify these facts. But rather than find a way to prove why this doesn't matter, why don't we assume it does? Then the knowledge and awareness might spread a little bit more through us, and if it is an issue, it might be that much more likely to be solved. There's no penalty for being wrong with a weakly held but well-meaning assumption. At the very least, we can be more aware of another possible dimension of bias in technology.

Re: Pulse oximeters give biased results for people with darker skin

#27
post #25

I'm a student EMT and our instructor told us not to rely on a pulse ox reading if our patient has dark skin, or if they were in a fire, or are a lifelong smoker, or if they have any nail polish or dirt/grease on their fingertips. We just look for the 94-100% range and would only take action if we get a reading around 87% or lower, or see a sudden drop during treatment. There are a few alternatives designs that aim to…

The article notes that the devices inaccurately report higher oxygen levels for darker skin tones. Meaning if someone dark skinned reports a result that falls into the 94-100% range, they may in fact be at lower oxygen levels that put them at risk.

That's true. There are other context clues that we can use, like a patient's medical history, how their breathing looks/sounds, or if their environment or mental status suggests CO poisoning (that also won't show up even on the perfect pulse ox, since it binds to hemoglobin the same way oxygen does). Overall we just need a better way to get information about oxygen levels.

Re: Pulse oximeters give biased results for people with darker skin

#28
post #13

This really isn't a big deal. Doctors know that these pulse ox meters aren't very accurate and use them accordingly. Nobody's changing their plan because you have a pulse ox of 93 instead of 95. They're there to quickly measure big changes that indicate a problem. In other words, they're looking for patients going from 98 to 75, not 98 to 95.

FTA: "At a reading of 88 or 89, Medicare will reimburse for oxygen at home, but at 90 it won’t"

Re: Pulse oximeters give biased results for people with darker skin

#29
And people with sweaty hands. And people with low iron levels. There's this push to show that everything is biased against certain classes. I'm sure some of the examples are true, but I can't imagine that constantly seeking evidence of problematic issues is good for individuals or society.

Re: Pulse oximeters give biased results for people with darker skin

#30
post #13

This really isn't a big deal. Doctors know that these pulse ox meters aren't very accurate and use them accordingly. Nobody's changing their plan because you have a pulse ox of 93 instead of 95. They're there to quickly measure big changes that indicate a problem. In other words, they're looking for patients going from 98 to 75, not 98 to 95.

That hasn’t been true at all in my experience. When my son got pneumonia, we were told he would be hospitalized at 92 when he was reading 94. He’s dark-skinned so who knows what the actual reading should have been.
Post reply on HN