Live data from Hacker News

Heart surgeons refuse difficult operations to avoid poor mortality ratings

telegraph.co.uk

191–200 of 274 posts

Re: Heart surgeons refuse difficult operations to avoid poor mortality ratings

#191
post #149
post #30

It's Goodhart's Law in action, this time with deadly consequences: "When a measure becomes a target, it ceases to be a good measure."[a] [a] https://en.wikipedia.org/wiki/Goodhart%27s_law

As a corollary, this is along the lines of one of my favorite bits of wisdom from Poor Charlie's Almanac. If I recall correctly, Munger considers incentives to be the single most important concept to properly understand in order to drive successful business (and arguably life) outcomes. Not surprisingly, incentives are also chronically underestimated or outright ignored, even in situations where there's a strong, pro…

For a different view, see "Punished by Rewards: The Trouble with Gold Stars, Incentive Plans, A's, Praise, and Other Bribes".[1]

It's about extrinsic vs intrinsic motivation. Extrinsic motivation comes from outside of the person and is what's typically used by organizations to try to motivate people: salary, praise, bonuses, etc. Intrinsic motivation comes from within.

[1] - https://www.amazon.com/Punished-Rewards-Trouble-Incentive-Pr...

Re: Heart surgeons refuse difficult operations to avoid poor mortality ratings

#192

Earlier quoted context omitted.

the problem here is that mortality rate is not correlated/normalized with the patient risk factor. the problem lies in the performance indicator itself, not strictly people trying to optimizing for what they are asked for. it is likely inhumane to refuse patients that have slightly above average risk - even if there are some considerations to be had, like having replacement organs go to waste, but that is already cov…

Another way of looking at it is that the patients who are refused treatment count as losses. So the current system is not actually maximizing wins, it's maximizing win percentage for doctors.

Ya, I went down this line of thinking too. It does seem like this might too be a poor incentive because now doctors would operate on everyone to try and claim wins, even if the likelihood is exceedingly low. There’s probably some measure like “healthy six months from now” that incentivizes them to try in situations with high mortality where operating is the only chance to improve things but keep them from operating if the risk is too high.

Re: Heart surgeons refuse difficult operations to avoid poor mortality ratings

#193
Metrics do drive behavior. Like, in the IT space, percentage of successful change requests. Encouraged you to power though even if things look bad. Because aborting is a 100% chance of a ding to the metric, while taking the risk is something less.

The stakes are lower, of course, but the idea is similar.

Re: Heart surgeons refuse difficult operations to avoid poor mortality ratings

#194
post #102

What would you prefer? A surgeon who expects a poor outcome of your surgery: a)decides to not operate on you? b)decides to operate on you nonetheless? I am in camp a

It depends what the outcome is without the surgery. If I'm dead without it, I want them to operate regardless. If there's only a chance of death, or the surgery is aiming to improve quality of life (rather than prevent [imminent] death), it's more of a judgement call.

Re: Heart surgeons refuse difficult operations to avoid poor mortality ratings

#195

If you’re a patient, wouldn’t this still mean starting with the lowest-mortality surgeon? If they take your case, you’re least likely to die. If not, you have a better appraisal of your odds.

I'm not so sure. That might mean going with a surgeon where you are at the top end of their "risk profile", rather than a more skilled surgeon who has higher mortality rates due to routinely taking more difficult cases. I'm not sure you can generalise the mortality rate of all patients to your specific chance of death (a more skilled surgeon would likely be better for you, c.p.).

Re: Heart surgeons refuse difficult operations to avoid poor mortality ratings

#196
post #60
post #32

Earlier quoted context omitted.

Yes, the system is horrendous and broken. There should be complete oversight and full transparency for self-regulation purposes, which includes ongoing learning and education, and get lawyers involved once the self-regulating is found to be inadequate - such as a surgeon who repeatedly makes the same mistakes and hasn't bothered to learn or care to improve their skills, or prevented from continuing. From my own strug…

This is one thing that still amazes me to this day. It's like doctor's stop thinking after they get their degree. Whenever I've visited a doc within the past few years I have noticed that instead of actually getting to know someone as a patient, many act as if they are a quick reference book. This is especially horrifying when you consider that there is a non-trivial portion of the population that exist outside of th…

> Nowadays, I'm usually making Doctors aware of advancements in their field... Scary stuff.

I don't find that to be too surprising. I can be hyper-aware of my own ailments, but keeping up with an entire field is more difficult. Furthermore, treating to research runs the risk of using treatments where the outcomes haven't been replicated, or there are long-term complications. Unless you've exhausted more conventional treatments/the treatment guidelines, there's something of an advantage of not being right at the bleeding edge.

Re: Heart surgeons refuse difficult operations to avoid poor mortality ratings

#197

Earlier quoted context omitted.

Some guy wrote a taxonomy of Goodhart's Law cases. This would be the "Adversarial Goodhart" case: "When you optimize for a proxy, you provide an incentive for adversaries to correlate their goal with your proxy, thus destroying the correlation with your goal." https://www.lesserwrong.com/posts/EbFABnst8LsidYs5Y/goodhart...

The first example on that page, "Regressional Goodhart", is totally wrong. If U measures V plus some noise X, assuming V and X form a bivariate normal distribution, then the conditional expectation of V is maximized by selecting the greatest U. In fact, the conditional expectation is linear in U. Not sure how much stock I can put into the rest of the article given this. Goodhart's Law can seemingly only apply if the…

> The first example on that page, "Regressional Goodhart", is totally wrong.

I don't think your statements contradict anything that it says. For reference, here are all the sentences in the "Quick Reference" part, which I'm assuming is the part you read.

Regressional Goodhart - When selecting for a proxy measure, you select not only for the true goal, but also for the difference between the proxy and the goal.

Model: When U is equal to V+X, where X is some noise, a point with a large U value will likely have a large V value, but also a large X value.

Thus, when U is large, you can expect V to be predictably smaller than U.

Example: height is correlated with basketball ability, and does actually directly help, but the best player is only 6'3", and a random 7' person in their 20s would probably not be as good

Is any particular one of these sentences false?

> If U measures V plus some noise X, assuming V and X form a bivariate normal distribution, then the conditional expectation of V is maximized by selecting the greatest U.

The word "maximized" carries some assumptions. If the only thing you can do is select based on U, then that statement is correct. However, if you had some means of selecting directly on V, then it's extremely likely that this would do better than selecting on U.

Suppose you're selecting the top 10 people. Suppose X ranges 0-10 chosen by a die roll, and V ranges 0-10, and it happens there are ten people with V=10, a hundred people with V=9, and a thousand with V=8 (and a lot more with lower Vs). On average, you'll have ten V=9s who score U=19, ten V=9s and a hundred V=8s who score U=18, and so on; in order for selecting on U to perform as well as selecting on V, every one of the V=10s would have to roll X=9 or X=10, which is exceedingly unlikely. It is true that taking people with high U scores yields people with higher Vs than taking people at random, and it is further true that taking the U=19s will give you better results than taking the U=18s or U=12s. It is simultaneously true that, when you take the U=19s, you'll be getting people whose X was 9 or 10, much higher than if you selected people at random or if you selected directly for high V. The first two sentences from the text state exactly this. (One consequence of this observation is, e.g., if you're doing admissions based on some test score, and you're considering raising the required score by ∆U, you should know the effect will be to raise average V and to raise average X, with ∆V=∆U-∆X, and if ∆X is large, you may be disappointed in the results. This is simple regression to the mean.)

I imagine you know all these concepts; I think you're interpreting the text as a stronger statement than it is.

Re: Heart surgeons refuse difficult operations to avoid poor mortality ratings

#198

Earlier quoted context omitted.

A large portion of the people undergoing heart surgeries still face a high risk of death if they don't have the surgery (with the surgery bringing the chance of a longer, more comfortable life). Here's why your hot take isn't the best take: “About 30 percent of them said they had turned patients down for surgery even when they knew full well that surgery was in their best interest.”

One problem is that fee-for-service incentives strongly distort what people think they "know full well."

Correct me if I am wrong, but don't doctors in the UK get paid a salary instead of the fee-for-service model prevalent in the US and Canada?

Re: Heart surgeons refuse difficult operations to avoid poor mortality ratings

#199
post #30

It's Goodhart's Law in action, this time with deadly consequences: "When a measure becomes a target, it ceases to be a good measure."[a] [a] https://en.wikipedia.org/wiki/Goodhart%27s_law

That only really applies when you're measuring side effects though, and not the actual thing intended. In this case they should have had a metric that took into account the estimated prior probability of a good outcome given an ordinary surgeon.

Mortality rate is a side effect.

Survival is the intended effect, but that's much more of a binary thing.

Re: Heart surgeons refuse difficult operations to avoid poor mortality ratings

#200
post #72
post #59

Earlier quoted context omitted.

Can someone really sue their surgeon in the US?

Absolutely. You can sue for pretty much anything. Winning is another story, but doctor's don't generally make sympathetic defendants. Juries know they have money and that they pay huge insurance fees to cover these cases. Of course, after losing a big case where the insurance company pays out millions of dollars, they likely become uninsurable and unemployable. But hey, the money is good until you hit the anti-lotter…

Let me guess, your heritage is Indian or Pakistani?
Post reply on HN