Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How one patient found errors in the algorithm making transplant decisions

ft.com

Post reply on HN
Print view View original

11–20 of 23 posts

Re: How one patient found errors in the algorithm making transplant decisions

#11

Post by orbz »

RecycledEle wrote: post #5
dpz wrote: post #4

Just finished reading this, I get why it's done but its really tragic and sad that an algorithm is deciding who lives or dies. Could you imagine working on something like that? I don't think I'd have the stomach.

You are correct. Those of us with moral values avoid this kind of work. Only the sociopaths do this kind of work. They only care about gaining power. Maybe genetically modified pig organs will eliminate the supply shortfall in organs and get rid of the power trip these jokers enjoy so much.

This feels unfair to say.

People with moral values don’t and shouldn’t avoid this kind of work to make sure that they care about accuracy, edge cases, that they continuously refine how they define utility and try to view it from a place of empathy.

Doctors and first responders make these calls often when they perform mass casualty triage, or when to declare someone’s death to enable collection of life saving organs.

Re: How one patient found errors in the algorithm making transplant decisions

#12

Post by drewcoo »

orbz wrote: post #11
RecycledEle wrote: post #5

Earlier quoted context omitted.

You are correct. Those of us with moral values avoid this kind of work. Only the sociopaths do this kind of work. They only care about gaining power. Maybe genetically modified pig organs will eliminate the supply shortfall in organs and get rid of the power trip these jokers enjoy so much.

This feels unfair to say. People with moral values don’t and shouldn’t avoid this kind of work to make sure that they care about accuracy, edge cases, that they continuously refine how they define utility and try to view it from a place of empathy. Doctors and first responders make these calls often when they perform mass casualty triage, or when to declare someone’s death to enable collection of life saving organs.

Doctors and first responders aren't faceless people in cubes making secret proprietary software for companies seeking to profit from the automated decisions.

Re: How one patient found errors in the algorithm making transplant decisions

#13

Post by isaacfrond »

As far as I can tell the algorithm was not flawed. At least not in the sense that is produced different outputs than intended. The algorithm was designed to favor older patients, and it did just that. The result was that older patients received more livers. As was the intention.

So it not an algorithm that is flawed, it is a policy that is flawed. A policy that flawlessly and fairly executed by algorithm exactly as it was designed.

I'm not saying the policy was bad or fair. I have no idea to be honest. If you have one liver and two patients, then it's always going to be hard choice. But I don't think it is helpful to say the algorithm was misbehaving when it was not.

In fact, as mentioned in article, the outcomes of the algorithm are regularly checked by humans. And when they found a genuine bug (misclassifying people with liver cancer) the algorithm was fixed. Isn't that more or less exactly what you want. Humans thinking about policy, then having a computer executing the policy, while humans regularly check its output to see if the algorithm aligns with the intention.

Re: How one patient found errors in the algorithm making transplant decisions

#14

Post by paulryanrogers »

isaacfrond wrote: post #13

As far as I can tell the algorithm was not flawed. At least not in the sense that is produced different outputs than intended. The algorithm was designed to favor older patients, and it did just that. The result was that older patients received more livers. As was the intention. So it not an algorithm that is flawed, it is a policy that is flawed. A policy that flawlessly and fairly executed by algorithm exactly as i…

Don't medical professionals strive to maximize years of life/health, at least in zerosum situations? Hence the whole triage thing.

Favoring older patients could be better at that or worse. But I'd suspect it's worse unless there is data proving otherwise.

Re: How one patient found errors in the algorithm making transplant decisions

#15

Post by Suzuran »

RecycledEle wrote: post #5
dpz wrote: post #4

Just finished reading this, I get why it's done but its really tragic and sad that an algorithm is deciding who lives or dies. Could you imagine working on something like that? I don't think I'd have the stomach.

You are correct. Those of us with moral values avoid this kind of work. Only the sociopaths do this kind of work. They only care about gaining power. Maybe genetically modified pig organs will eliminate the supply shortfall in organs and get rid of the power trip these jokers enjoy so much.

The "supply shortfall" is due to the requirement that the DNA of the organ has to match up with the DNA of the recipient in order for the organ to function and the fact that the organ has a very short "shelf life" when outside of the donor's body. You can't just randomly stuff organs into people and expect a good outcome, and you can't just stockpile them.

Until we can customize the DNA in the organ for the intended recipient and manufacture them on demand with a short lead time, we are going to have some kind of lack of availability issue. Our issues are not a bunch of power-tripping sociopaths sitting on a hoard of organs deciding who lives or dies, they are trying to rapidly identify who is physically close enough and who genetically matches in the hours you have when an organ becomes available, and is there any opportunity for a cascade donation to be brought into the mix (is this organ compatible with someone who has a willing but incompatible donor who is compatible with someone else, and does that person have a willing but incompatible donor, and so on down the line)

(Edit: I have been informed that the paywalled article, which I cannot read, is about a UK system; I am only familiar with the kidney system in the US. This post probably does not reflect UK practice)

Re: How one patient found errors in the algorithm making transplant decisions

#16

Post by bitshiftfaced »

paulryanrogers wrote: post #14
isaacfrond wrote: post #13

As far as I can tell the algorithm was not flawed. At least not in the sense that is produced different outputs than intended. The algorithm was designed to favor older patients, and it did just that. The result was that older patients received more livers. As was the intention. So it not an algorithm that is flawed, it is a policy that is flawed. A policy that flawlessly and fairly executed by algorithm exactly as i…

Don't medical professionals strive to maximize years of life/health, at least in zerosum situations? Hence the whole triage thing. Favoring older patients could be better at that or worse. But I'd suspect it's worse unless there is data proving otherwise.

That's how I'd like to think that they'd go about it. I'd be interested in learning the details. Say for example that younger people tend to be able to go without for a longer period than older people. I don't know if that's true, but if so you could prolong the lives of older people now but then get to the younger folks later, but not necessarily vice versa.

Re: How one patient found errors in the algorithm making transplant decisions

#17

Post by dado3212 »

RecycledEle wrote: post #5
dpz wrote: post #4

Just finished reading this, I get why it's done but its really tragic and sad that an algorithm is deciding who lives or dies. Could you imagine working on something like that? I don't think I'd have the stomach.

You are correct. Those of us with moral values avoid this kind of work. Only the sociopaths do this kind of work. They only care about gaining power. Maybe genetically modified pig organs will eliminate the supply shortfall in organs and get rid of the power trip these jokers enjoy so much.

Or an alternative framing: using data can help to remove human biases and increase overall lifespans (which the studies have shown). How is it sociopathic to work on software that demonstrably saves lives?

Re: How one patient found errors in the algorithm making transplant decisions

#18

Post by dghlsakjg »

dpz wrote: post #4

Just finished reading this, I get why it's done but its really tragic and sad that an algorithm is deciding who lives or dies. Could you imagine working on something like that? I don't think I'd have the stomach.

It sounds like the algorithm in question was designed by a committee of experts to make a better triage process. Given that the previous process for deciding who gets an organ sounded a lot like individual doctors just picking patients based on proximity and how good the patient advocacy was, I think this is a much fairer alternative.

At the end of the day, triage is always a hard thing to do, but it is based in the idea of optimizing for the best global outcome at the expense of specific individual outcomes.

I personally would have no qualms about implementing the algorithm design in code (the organ score is going to be computed by hand or computed by code regardless, might as make a tool that saves time and mistakes), and would have no problem participating in the algorithm design if I had the appropriate expertise.

At the end of the day I'd rather live in a world where there is a known, predictable process for these decisions instead of an informal network of professionals making game-time decisions.

Re: How one patient found errors in the algorithm making transplant decisions

#19

Post by sodii »

JudasGoat wrote: post #7
dpz wrote: post #4

Just finished reading this, I get why it's done but its really tragic and sad that an algorithm is deciding who lives or dies. Could you imagine working on something like that? I don't think I'd have the stomach.

At a recent physical, my doctor informed me that "according to the drug company's algorithm" I should begin taking a cholesterol reducing drug. I had to wonder about the transparency of such an approach.

'algorithm' in that context is not a computer program, but a flow chart or decision tree made by distilling the result of various medical trials.

Re: How one patient found errors in the algorithm making transplant decisions

#20

Post by lovemenot »

bitshiftfaced wrote: post #16
paulryanrogers wrote: post #14

Earlier quoted context omitted.

Don't medical professionals strive to maximize years of life/health, at least in zerosum situations? Hence the whole triage thing. Favoring older patients could be better at that or worse. But I'd suspect it's worse unless there is data proving otherwise.

That's how I'd like to think that they'd go about it. I'd be interested in learning the details. Say for example that younger people tend to be able to go without for a longer period than older people. I don't know if that's true, but if so you could prolong the lives of older people now but then get to the younger folks later, but not necessarily vice versa.

Quality Adjusted Life Year (QALY) is used in the UK.

https://en.wikipedia.org/wiki/Quality-adjusted_life_year

Post reply on HN