Live data from Hacker News

Heart surgeons refuse difficult operations to avoid poor mortality ratings

telegraph.co.uk

121–130 of 274 posts

Re: Heart surgeons refuse difficult operations to avoid poor mortality ratings

#121
post #30

It's Goodhart's Law in action, this time with deadly consequences: "When a measure becomes a target, it ceases to be a good measure."[a] [a] https://en.wikipedia.org/wiki/Goodhart%27s_law

The version I've heard might be a corollary: if you measure it, it will improve.

The subtext is that the system won't improve; rather, something in the system will be sacrificed to improve the metric. The end result (usually quality) won't improve.

Re: Heart surgeons refuse difficult operations to avoid poor mortality ratings

#122
post #90

They need to add a difficulty correlated metric to the statistics. Maybe BMI and age? I am sure IVF clinics wouldn't offer the procedure to anyone over 35 if the rather detailed statistics they are required to report didn't also include the patient's age.

I imagine it’s extremely difficult to come up with a metric that captures everything about the case. BMI, age, tumor type, tumor shape, history of kidney disease, diabetes, where do you stop?

You don't need a metric which captures everything about the case - you just need a statistical model whose risk assessment is at least as accurate as the surgeon's assessment. This is not as difficult as it sounds - in Chapter 21 of Thinking Fast and Slow, Kahneman makes a strong case that simple algorithms are very often better than expert clinical judgement.

Re: Heart surgeons refuse difficult operations to avoid poor mortality ratings

#123

Sports athletes do this. For example players won't attempt end of the quarter half court shots since that will lower their percentages, or QBs taking sacks to not lower their QB ratings. What they do in sports is keep advanced stats. Instead of just giving a pure mortality rating have mortality rating, by surgery type.

Maybe in other sports this happens, but I seriously doubt QB's engage in that. First of all, taking a sack is not pleasant. In the extreme it can take you out of the game, or land you on the IR (injured reserve). Drew Bledsoe was replaced by Tom Brady after such a sack, and was never the star he had been before. Second, the number of sacks a QB takes is a statistic in itself. Third, the top stats for a QB are in this…

some QBs like aaron rodgers will take a sack instead of throwing the ball away.

rodgers had a 50 sack season twice in his career and has a sack career of 6.96. this is reflected on his interception rate (which is the best in the nfl). he just prefers to take a sack to attempt a pass that could be intercepted -- and everybody looks at those int/td numbers and don't look a lot at sack numbers (because sacks are usually not the qb fault).

Re: Heart surgeons refuse difficult operations to avoid poor mortality ratings

#124

Earlier quoted context omitted.

I'm not so sure Goodhart's Law is correct. For example, if I target a high win percentage, is win percentage, then, not a good measure? Mortality rates are a pretty good measure for surgeons. The problem here is that they are inflating their measure by cherry picking. edit: Some measures should be targeted, the problem here is that it is done in an non genuine way

the problem here is that mortality rate is not correlated/normalized with the patient risk factor. the problem lies in the performance indicator itself, not strictly people trying to optimizing for what they are asked for. it is likely inhumane to refuse patients that have slightly above average risk - even if there are some considerations to be had, like having replacement organs go to waste, but that is already cov…

Another way of looking at it is that the patients who are refused treatment count as losses. So the current system is not actually maximizing wins, it's maximizing win percentage for doctors.

Re: Heart surgeons refuse difficult operations to avoid poor mortality ratings

#125
The incorrect assumption your readers are making is that heart surgery is a good thing in all cases. America is the leader in futile medical procedures: it may well be a good thing to have fewer risky procedures on patients who were more likely to be harmed. "primum no nocere" may be advanced!

Re: Heart surgeons refuse difficult operations to avoid poor mortality ratings

#126

Earlier quoted context omitted.

I'm not so sure Goodhart's Law is correct. For example, if I target a high win percentage, is win percentage, then, not a good measure? Mortality rates are a pretty good measure for surgeons. The problem here is that they are inflating their measure by cherry picking. edit: Some measures should be targeted, the problem here is that it is done in an non genuine way

the problem here is that mortality rate is not correlated/normalized with the patient risk factor. the problem lies in the performance indicator itself, not strictly people trying to optimizing for what they are asked for. it is likely inhumane to refuse patients that have slightly above average risk - even if there are some considerations to be had, like having replacement organs go to waste, but that is already cov…

Physicians already have good reasons to overstate the patient's severity. It's part of the payment formula used by CMS. But there are plenty of incentives not to lie such as jail time.

Re: Heart surgeons refuse difficult operations to avoid poor mortality ratings

#127
post #30

It's Goodhart's Law in action, this time with deadly consequences: "When a measure becomes a target, it ceases to be a good measure."[a] [a] https://en.wikipedia.org/wiki/Goodhart%27s_law

Thanks. A few years ago I asked a question on what this was called.

Re: Heart surgeons refuse difficult operations to avoid poor mortality ratings

#128
This exact topic was discussed in the 1968 novel Case of Need by Jeffrey Hudson. It was his first book, written while a medical student at Harvard. The novel raised a stir as it was critical of Harvard faculty and in some ways the practice of medicine in general. The book eventually won an Edgar Award.

Some may know Jeffrey Hudson better by his given name, Michael Crichton.

https://en.m.wikipedia.org/wiki/A_Case_of_Need

Re: Heart surgeons refuse difficult operations to avoid poor mortality ratings

#129
post #30

It's Goodhart's Law in action, this time with deadly consequences: "When a measure becomes a target, it ceases to be a good measure."[a] [a] https://en.wikipedia.org/wiki/Goodhart%27s_law

When I was fresh out of a school I worked in a call center for a while. One of the most important metrics (and everything you did was measured) was the average length of each call. So it was common practice, when you got cases that basically were going to be long calls or ones you probably couldn't actually resolve (e.g., "I want the service back on but cannot pay"), to deliberately upset the customers so that they w…

I'd also expect a bunch of "I'm going to transfer you... oops, pressed a wrong button and it hang up!" As a customer, I've had a fair measure of mysteriously hang up calls to support, I wonder how many of those are metrics-driven...

Re: Heart surgeons refuse difficult operations to avoid poor mortality ratings

#130

Earlier quoted context omitted.

What are the rates?

http://www.mercyforanimals.org/american-heart-association-ve...

> Researchers studied five different diets to see which lowered the risk of heart failure: convenience (fast food and pasta), plant-based, sweets, Southern (sugary beverages and fried foods), and alcohol/salads.

What kind of study design is this? It seems like they compared veganism with 4 garbage diets. Seems like all we can extract from this is plant-based is better than eating trash.

Post reply on HN