[deleted]
OpenAI’s o1 correctly diagnosed 67% of ER patients vs. 50-55% by triage doctors
351–360 of 500 posts
Re: OpenAI’s o1 correctly diagnosed 67% of ER patients vs. 50-55% by triage doctors
#352Earlier quoted context omitted.
You've witnessed a dismantle and replace effort by the right wing that wishes to squeeze everything to make rich people more money. An effort by the left would destroy the private insurance scheme and build up medicare. Completely different and you'd get something functional. When the wrong targets get destroyed, everyone suffers. When parasitic forces are destroyed, the system functions better. It's the difference b…
We already had an effort by the left. You can “no true scotsmen” if you want, but it represents the reality of what will happen when ideals clash a sector that makes up 18% of the GDP. What’s going to be different now than in 2010?
[dead]
Re: OpenAI’s o1 correctly diagnosed 67% of ER patients vs. 50-55% by triage doctors
#353Earlier quoted context omitted.
It’s the ER. People aren’t always in a position to “chat” when they go there.
You think current ER people work in complete silence? No words uttered?
You think that they have “days leading up to consultation”? Please don’t be so disingenuous; I’m sure you know exactly what the person you’re replying to meant.
Re: OpenAI’s o1 correctly diagnosed 67% of ER patients vs. 50-55% by triage doctors
#354I'd be very very hesitant to trust studies like this. It's very easy to mess up these benchmarks. See for example this recent paper where AI managed to beat radiologists on interpreting x-rays... when the AI didn't even have access to the x-rays: https://arxiv.org/pdf/2603.21687 (on a pre existing "large scale visual question answering benchmark for generalist chest x-ray understanding" that wasn't intentionally mess…
I agree with you on this specific study, however, I can't really wrap my head about the fact that doctors will be better than AI models on the long-run. After all, medicine is all about knowledge, experience and intelligence (maybe "pattern recognition"), all those, we must assume that the best AI models (especially ones focusing solely in the medical field) would largely beat large majority of humans (aka doctors),…
> What is the specific capability (or combination of capabilities)
The ability to go to prison / be stripped of a license when something goes wrong.
A single doctor will care for far fewer patients in their career than an AI system will. Even if the AI system is 10x less likely to make mistakes, the sheer number of patients will make it much more likely to make a mistake somewhere.
With a single doctor, the PR and legal fallout of a medical error is limited to that doctor. This preserves trust in the medical system. The doctor made a mistake, they were punished, they're not your doctor, so you're not affected and can still feel safe seeing whoever you're seeing. AI won't have that luxury.
Re: OpenAI’s o1 correctly diagnosed 67% of ER patients vs. 50-55% by triage doctors
#355> "An AI and a pair of human doctors were each given the same standard electronic health record to read" This is handicapping the human doctors abilities. There is a lot more information a human doctor can gather even with a brief observation of the patient.
They have covered this in the article.
> But it is not curtains for emergency doctors yet, the researchers said. The study only tested humans against AIs looking at patient data that can be communicated via text. The AI’s reading of signals, such as the patient’s level of distress and their visual appearance, were not tested. That means the AI was performing more like a clinician producing a second opinion based on paperwork.
Re: OpenAI’s o1 correctly diagnosed 67% of ER patients vs. 50-55% by triage doctors
#356Earlier quoted context omitted.
I agree with you on this specific study, however, I can't really wrap my head about the fact that doctors will be better than AI models on the long-run. After all, medicine is all about knowledge, experience and intelligence (maybe "pattern recognition"), all those, we must assume that the best AI models (especially ones focusing solely in the medical field) would largely beat large majority of humans (aka doctors),…
> What is the specific capability (or combination of capabilities) The ability to go to prison / be stripped of a license when something goes wrong. A single doctor will care for far fewer patients in their career than an AI system will. Even if the AI system is 10x less likely to make mistakes, the sheer number of patients will make it much more likely to make a mistake somewhere. With a single doctor, the PR and le…
> > What is the specific capability (or combination of capabilities)
> The ability to go to prison / be stripped of a license when something goes wrong.
So basically you need a person to blame if things don't go the best way possible?
Re: OpenAI’s o1 correctly diagnosed 67% of ER patients vs. 50-55% by triage doctors
#357I'm surprised at both the article and the paper - both seem very hyperbolic. This is LLMs competing against doctors in a way that is heavily weighted in the LLMs favour, which does not represent clinical practice. These reasoning cases are not benchmarks for doctors, they are learning tools. I think it's important to note that diagnosis also relies on accurate description of the patient in the first place, and the in…
Also, you need to see an analysis of the incorrect calls. The goal of a human Dr is not to get the highest accuracy, it's to limit total harm to the patient. There can be cases where the odds favor picking X (but it may not be by that much), but the safe thing to do is to rule out some other option first, or start a safe treatment that covers several other possible options. Simply getting the "high score" on this eva…
Exactly this. Most diagnosis isn’t about pinpointing the underlying exact cause, it’s ruling out the really bad stuff and minimising harm. Differential diagnosis just isn’t real world medicine.
Re: OpenAI’s o1 correctly diagnosed 67% of ER patients vs. 50-55% by triage doctors
#358Earlier quoted context omitted.
I agree with you on this specific study, however, I can't really wrap my head about the fact that doctors will be better than AI models on the long-run. After all, medicine is all about knowledge, experience and intelligence (maybe "pattern recognition"), all those, we must assume that the best AI models (especially ones focusing solely in the medical field) would largely beat large majority of humans (aka doctors),…
To answer your question: talking to a human. Medicine is so much more than "knowledge, experience, and pattern matching", as any patient ever can attest to. Why is it so hard for some people to understand that humans need other humans and human problems can't be solved with technology?
In psychotherapy patients tend to prefer talking to AI than a human therapist and rank the interaction higher.
Re: OpenAI’s o1 correctly diagnosed 67% of ER patients vs. 50-55% by triage doctors
#359o1 is several generations old and was released in 2024. Is this some quite old research that took a long time to get published?
It's hard to draw any conclusion from this study precisely because of this. Since 2024, we went from AI being able to do a few minutes of coding work to now a few weeks autonomously. That's like going from an intern to staff engineer level.
Re: OpenAI’s o1 correctly diagnosed 67% of ER patients vs. 50-55% by triage doctors
#360Earlier quoted context omitted.
So much of what I know from women in my life is that the human element of medicine is almost a strict negative for them. As a guy it hasn't been much better, but at least doctors listen to me when I say something.
One of, if not THE biggest challenge in getting treatment is getting past insurance rules designed to deny treatment. This is much, much easier when you're able to convince a doctor (and/or trained medical staff) to argue on your behalf. If you can't get those folks to listen to you, that's probably not gonna happen. You might have to go through several different practices before you find a sympathetic ear. Now repla…
The whole system has basic flaws in how's financing set up.
There is an intermediary between customers and seller and it's allowed to take percentage of the sale. No such entity will ever work in the interest of the consumer. It has every incentive to inflate prices. Intermediary is needed but it should be financed by buyers with flat fee (possibly for additional incentives that reinforce the desired behavior). The tragedy here is that initially it was. But it was deemed too expensive for the buyers and got privatized which made it vastly more expensive in the long run.
Insurance is also wrong. Insurance is gambling and gambling needs restrictions. You are allowed to take people's money without providing any service most of the time, so you shouldn't be allowed to refuse legal service for that privilege.