Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI’s o1 correctly diagnosed 67% of ER patients vs. 50-55% by triage doctors

theguardian.com

Post reply on HN
Print view View original

241–250 of 500 posts

Re: OpenAI’s o1 correctly diagnosed 67% of ER patients vs. 50-55% by triage doctors

#241

Post by dghlsakjg »

sdwr wrote: post #224
hyperpape wrote: post #200

Earlier quoted context omitted.

> we must assume that the best AI models (especially ones focusing solely in the medical field) would largely beat large majority of humans (aka doctors), if we already have this assumption for software engineers, we should have it for this field as well, This is a pretty wild leap. Code has a lot of hooks for training via hill-climbing during post-training. During post-training, you can literally set up arbitrary sc…

Emergency medicine is the coding of medicine. Fast feedback loop, requires broad rather than deep judgement, concrete next steps. The AI coding improvement should be partially transferrable to other disciplines without recreating the training environment that made it possible in the first place. The model itself has learned what correct solutions "feel like", and the training process and meta-knowledge must have impr…

I would argue that the ED is the least similar to code. You have the most unknowns, unreliable data and history, non deterministic options and time constraints.

An ER staff is frequently making inferences based on a variety of things like weather, what the pt is wearing, what smells are present, and a whole lot of other intangibles. Frequently the patients are just outright lying to the doctor. An AI will not pick up on any of that.

Re: OpenAI’s o1 correctly diagnosed 67% of ER patients vs. 50-55% by triage doctors

#242

Post by [deleted] »

creativeSlumber wrote: post #36

> "An AI and a pair of human doctors were each given the same standard electronic health record to read" This is handicapping the human doctors abilities. There is a lot more information a human doctor can gather even with a brief observation of the patient.

[deleted]

Re: OpenAI’s o1 correctly diagnosed 67% of ER patients vs. 50-55% by triage doctors

#243

Post by ipaddr »

phoronixrly wrote: post #127
ipaddr wrote: post #121

Earlier quoted context omitted.

Doctors are not necessarily great at talking to patients and patients are unhappy with the information Doctors provide. This moat has dried up.

If you prefer an LLM to a human doctor, you deserve an LLM instead of a human doctor, and I wish you get it.

I would use one for sure. Much of medicine is getting tests / labs booked fighting to get certain medicines. Doctors will barely give you 5 minutes only deal with one issue per visit, rarely are available and going into an office can make you sicker. An llm with Doctor powers could offer more. I don't think we are at the surgery point but we are past getting notes and medicine's refilled.

Re: OpenAI’s o1 correctly diagnosed 67% of ER patients vs. 50-55% by triage doctors

#244

Post by tensor »

gpm wrote: post #11

I'd be very very hesitant to trust studies like this. It's very easy to mess up these benchmarks. See for example this recent paper where AI managed to beat radiologists on interpreting x-rays... when the AI didn't even have access to the x-rays: https://arxiv.org/pdf/2603.21687 (on a pre existing "large scale visual question answering benchmark for generalist chest x-ray understanding" that wasn't intentionally mess…

Interestingly, this recent study using ChatGPT Health gave quite a different outcome (https://www.nature.com/articles/s41591-026-04297-7). Here it was wrong about emergency triage 50% of the time.

Re: OpenAI’s o1 correctly diagnosed 67% of ER patients vs. 50-55% by triage doctors

#245

Post by TurdF3rguson »

dghlsakjg wrote: post #241
sdwr wrote: post #224

Earlier quoted context omitted.

Emergency medicine is the coding of medicine. Fast feedback loop, requires broad rather than deep judgement, concrete next steps. The AI coding improvement should be partially transferrable to other disciplines without recreating the training environment that made it possible in the first place. The model itself has learned what correct solutions "feel like", and the training process and meta-knowledge must have impr…

I would argue that the ED is the least similar to code. You have the most unknowns, unreliable data and history, non deterministic options and time constraints. An ER staff is frequently making inferences based on a variety of things like weather, what the pt is wearing, what smells are present, and a whole lot of other intangibles. Frequently the patients are just outright lying to the doctor. An AI will not pick up…

> An AI will not pick up on any of that.

It will if it trains on data like that. It's all about the training data.

Re: OpenAI’s o1 correctly diagnosed 67% of ER patients vs. 50-55% by triage doctors

#246

Post by djb_hackernews »

creativeSlumber wrote: post #36

> "An AI and a pair of human doctors were each given the same standard electronic health record to read" This is handicapping the human doctors abilities. There is a lot more information a human doctor can gather even with a brief observation of the patient.

Can't the same be said for the AI?

Re: OpenAI’s o1 correctly diagnosed 67% of ER patients vs. 50-55% by triage doctors

#248

Post by NiloCK »

prmph wrote: post #125
ddosmax556 wrote: post #117

Earlier quoted context omitted.

I think there's a real space there, and a lot of what e.g. nurses and doctors do is talking to humans, and that won't go away. But two facts are also true: a) diagnosis itself can be automated. A lot of what goes on between you having an achy belly and you getting diagnosed with x y or z is happening outside of a direct interaction with you - all of that can be augmented with AI. And b), the human interaction part is…

> Diagnosis becomes cheaper & easier -> more time to actually talk to patients Unfortunately is this not likely to happen. More like: Diagnosis becomes cheaper & easier -> more patients a doctor is expected to see in the same period of time as before

What's unfortunate about that?

Re: OpenAI’s o1 correctly diagnosed 67% of ER patients vs. 50-55% by triage doctors

#249

Post by llbbdd »

voxl wrote: post #223
llbbdd wrote: post #194

Earlier quoted context omitted.

Lol, sharing specifics famously a comfortable and smart thing to do with medical information, doctor. This kind of attitude is why the moment it's viable, every F-student with a doctorate is going to get what they deserve.

What do they deserve?

The opportunity to compete with an autocomplete engine that does their job better on average, instead of coasting on their credentials and hurting real people in the process.

Re: OpenAI’s o1 correctly diagnosed 67% of ER patients vs. 50-55% by triage doctors

#250

Post by mitkebes »

namuol wrote: post #110
brikym wrote: post #106

Earlier quoted context omitted.

I think it's plausible since doctors tend to have human cognitive biases and miss things. People tend to fixate on patterns they're most familiar with.

A bold claim to suggest that LLMs aren’t prone to biases of their own which are less understood.

LLMs are having pretty consistent studies into their biases. Obviously this doesn't mean we know all the biases, but it's being actively worked on.

Meanwhile with human doctors, every one of them is a unique person with a completely different set of biases. In my experience, getting a correct diagnosis or treatment plan often involves trying multiple doctors, because many of them will jump to a common diagnosis even if the symptoms don't line up and the treatment doesn't actually help.

Post reply on HN