Earlier quoted context omitted.
I could see that problem occurring if the metric was “what is the success rate of everything that Surgeon X does”. I can’t see that problem occurring for “Surgeon X performing Procedure Y has N% of patients reporting relief and M% of patients reporting complications after the surgery”. What am I missing here? Edit: Follow-up question: notwithstanding the dysfunction of congress and the ability of companies to find lo…
I'm guessing you haven't heard of Goodhart's Law? ( https://en.wikipedia.org/wiki/Goodhart%27s_law ) Under your proposal, surgeons will be incentivized to selectively operate on easier patients and minimize their complication rates while not performing surgery on very sick patients who may also need the same surgery. Different surgeons in different areas treat different kinds of patients. It's hard to accurately meas…
To your counter-example, maybe the metrics I described aren’t good enough and should integrate some disease severity criteria or site-weighting or comorbidity score (although as one continues to subdivide the population this way eventually you end up with n=1 and the results are useless again) but like, surely we should be trying to measure something other than the good feels and word-of-mouth of people who have to work with each other?
It physically hurts my brain when I think about how we measure the dumbest shit in software engineering, like which shade of blue to use to improve clickthroughs[0], but when it comes to even attempting quantification of activities which are literally life or death, sorry, too hard, can’t do it. Surgeons will refuse to do hard procedures, insurers will destroy careers, EMRs are full of bad data (so what is the point of the bloody records if they have become that useless‽), surveying patients would cost too much…
I will ultimately defer to the experience of people in the field—I am not a physician or statistician—but sometimes I feel like I’m just being fed arguments repurposed from the bad cops playbook. Oh, we can’t ever possibly start quantifying individual officers’ use of force, because some parts of the city have more crime, and if we do that then those officers will look worse, so they will stop responding to violent calls in those areas, and there’ll be even more crime, so get off our backs man and stop trying to create more objective metrics for accountability.
To be clear I don’t think you are arguing in bad faith and I don’t intend my statement about accountability to suggest that you personally are trying to avoid it or shield bad actors or anything. What you are saying is probably true and I may be wrong to challenge it at all since I have no personal insight into what is going on behind the scenes, and I genuinely appreciate you answering my questions from your perspective and giving me additional perspectives and things to think about. It just feels so, so frustrating as a patient. All I want is some ability to measure risk that’s better than looking up studies on procedure X on pubmed that I’m unqualified to interpret (and which don’t apply anyway because the lead author of the research won’t be doing my procedure), or shaking the magic eight ball.
If I were a physician, I would absolutely want to track the shit out of my own patient outcomes so I could improve, and the amount of resistance that seems to exist (this is not the first time I’ve talked to docs about this and received similar fatalistic answers) is just baffling to me.
We’re not talking about Frogger here, metrics aren’t some high score, if you have an 80% complication rate for some procedure that isn’t necessarily a reflection on you as a practitioner but it would suggest that there is a problem that needs to be identified (bad procedure, bad training, bad support, bad patient, bad luck). Right now, it seems like no one really knows.
This isn’t bullshit alternative medicine, so why, when I scratch beneath the surface, does it so often feel like it is anyway?