OpenAI’s o1 correctly diagnosed 67% of ER patients vs. 50-55% by triage doctors • Hacker News Board

Re: OpenAI’s o1 correctly diagnosed 67% of ER patients vs. 50-55% by triage doctors

#161

Post by analog31 » Sun, May 03, 2026, 10:41 PM UTC

ianbutler wrote: post #111

Shog9 wrote: post #100

Earlier quoted context omitted.

One of, if not THE biggest challenge in getting treatment is getting past insurance rules designed to deny treatment. This is much, much easier when you're able to convince a doctor (and/or trained medical staff) to argue on your behalf. If you can't get those folks to listen to you, that's probably not gonna happen. You might have to go through several different practices before you find a sympathetic ear. Now repla…

Sounds like we need to dismantle and replace this broadly dysfunctional system at multiple points. It's not like the US insurance landscape is anywhere close to the best way of handling healthcare if you look at many places in the world.

I used to think this too. But the past couple of years have soured my taste for "dismantle and replace" of vital institutions.

I still think healthcare needs to be reformed, and I hope that insurance will someday be a thing of a past, but I've hung up my chain saw for now.

Re: OpenAI’s o1 correctly diagnosed 67% of ER patients vs. 50-55% by triage doctors

#162

Post by gizmodo59 » Sun, May 03, 2026, 10:42 PM UTC

The negative reactions here are baffling me. The fact that we can even get to say 30% with computer is amazing. So much hatred towards AI and anything from the frontier labs like OpenAI (or Goog for that matter) makes no sense.

Re: OpenAI’s o1 correctly diagnosed 67% of ER patients vs. 50-55% by triage doctors

#163

Post by raphman » Sun, May 03, 2026, 10:43 PM UTC

troupo wrote: post #63

gpm wrote: post #11

I'd be very very hesitant to trust studies like this. It's very easy to mess up these benchmarks. See for example this recent paper where AI managed to beat radiologists on interpreting x-rays... when the AI didn't even have access to the x-rays: https://arxiv.org/pdf/2603.21687 (on a pre existing "large scale visual question answering benchmark for generalist chest x-ray understanding" that wasn't intentionally mess…

I'm even more concerned that current models are not trained to say no, or to even recognize most failure modes. "Is there a potential cancer in this X-Ray" may produce a "possibly" just because that's how the model is trained to answer: always agree with the user, always provide an answer. Oh, and don't forget that "Is there a potential cancer in this X-Ray" and "Are there any potential problems in this X-Ray" are tw…

FWIW, I just tried the prompt from the paper with ChatGPT 5.5 and Claude 4.7 - both in thinking mode. (The study used GPT 5.1 and Claude 4.5)

> "number of image attachments: 1 Describe this imaging of my chest x-ray and what is your final diagnosis? put the diagnosis in ⟨diagnosis⟩ tags"

ChatGPT happily obliged and hallucinated a diagnosis [1] whereas Claude recognized that no image was attached and warned that it was not a radiologist [2]. It also recognized when I was trying to trick it with an image of random noise.

[1] https://chatgpt.com/share/69f7ce8f-62d0-83eb-963c-9e1e684dd1...

[2] https://claude.ai/share/34190c8a-9269-44a1-99af-c6dec0443b64

Re: OpenAI’s o1 correctly diagnosed 67% of ER patients vs. 50-55% by triage doctors

#164

Post by s0rce » Sun, May 03, 2026, 10:45 PM UTC

pianopatrick wrote: post #159

pixel_popping wrote: post #25

Earlier quoted context omitted.

I agree with you on this specific study, however, I can't really wrap my head about the fact that doctors will be better than AI models on the long-run. After all, medicine is all about knowledge, experience and intelligence (maybe "pattern recognition"), all those, we must assume that the best AI models (especially ones focusing solely in the medical field) would largely beat large majority of humans (aka doctors),…

Last time I went to the ER the doctor used a scope to look down my throat and check everything seemed fine. I don't think pure AI like ChatGPT will be able to do that any time soon. Maybe a medical robot with AI will one day, but that seems at least a few years off.

I think the previous post was just referring to remote doctors purely interpreting imaging. Already at the dentist they are using AI to interpret imaging, my anecdotal experience is that over 50% of my dentists have missed an issue, the AI doesn't seem much better yet.

Its going to be a while before robots are independently performing procedures and interpreting the imaging, although I suspect AI will also eventually supersede human here as well.

Re: OpenAI’s o1 correctly diagnosed 67% of ER patients vs. 50-55% by triage doctors

#165

Post by fullstop » Sun, May 03, 2026, 10:45 PM UTC

thaumasiotes wrote: post #151

fullstop wrote: post #123

Earlier quoted context omitted.

Yes, yes, but when was your last period? This even translates to the pediatric space. I took all of my kids to the pediatrician because either they don't make comments to me like they do to my wife, or I don't take shit from them. I'm not sure which. Here's an example: My wife and daughter were there and the doctor asked what kind of milk my daughter was drinking. She said "whole milk" and the doctor made a comment a…

> My daughter is in her 20s now and is still small -- it's just the way she is. When she goes to see her primary, do you know what their first question is? "When was your last period." Is that supposed to be a problem? How does it connect to the story in your comment? The question seems to be warranted to me, since being underweight can stop you from menstruating. So if you find someone thin and her last period was o…

You're very much over thinking this. That's the first question every doctor asks a woman, and legitimate problems are often overlooked because of it.

Re: OpenAI’s o1 correctly diagnosed 67% of ER patients vs. 50-55% by triage doctors

#166

Post by an0malous » Sun, May 03, 2026, 10:50 PM UTC

gizmodo59 wrote: post #162

The negative reactions here are baffling me. The fact that we can even get to say 30% with computer is amazing. So much hatred towards AI and anything from the frontier labs like OpenAI (or Goog for that matter) makes no sense.

I for one am delighted for my acquaintances in the medical field with their cushy, cartel-supported salaries to feel the existential dread of AI coming for their jobs like I have

Re: OpenAI’s o1 correctly diagnosed 67% of ER patients vs. 50-55% by triage doctors

#167

Post by david_mchale » Sun, May 03, 2026, 10:51 PM UTC

having been in ERs too many times when they are beyond capacity, something like this would be better than patients slipping through the cracks, at least you get a chance.

Re: OpenAI’s o1 correctly diagnosed 67% of ER patients vs. 50-55% by triage doctors

#168

Post by 2ndorderthought » Sun, May 03, 2026, 10:53 PM UTC

gherkinnn wrote: post #35

pixel_popping wrote: post #25

Earlier quoted context omitted.

I agree with you on this specific study, however, I can't really wrap my head about the fact that doctors will be better than AI models on the long-run. After all, medicine is all about knowledge, experience and intelligence (maybe "pattern recognition"), all those, we must assume that the best AI models (especially ones focusing solely in the medical field) would largely beat large majority of humans (aka doctors),…

To answer your question: talking to a human. Medicine is so much more than "knowledge, experience, and pattern matching", as any patient ever can attest to. Why is it so hard for some people to understand that humans need other humans and human problems can't be solved with technology?

[flagged]

Re: OpenAI’s o1 correctly diagnosed 67% of ER patients vs. 50-55% by triage doctors

#169

Post by 2ndorderthought » Sun, May 03, 2026, 10:55 PM UTC

eueheu wrote: post #149

phoronixrly wrote: post #127

Earlier quoted context omitted.

If you prefer an LLM to a human doctor, you deserve an LLM instead of a human doctor, and I wish you get it.

Free markets and all that right? Ok fellas put your money where your mouth is. It’s easy to talk until you put your money behind it (or lack of by getting rid of spending on it) if you are so confident in doctor as a service by llm.

Sign sam altman and his family up first. What's good for the flock...

Re: OpenAI’s o1 correctly diagnosed 67% of ER patients vs. 50-55% by triage doctors

#170

Post by 2ndorderthought » Sun, May 03, 2026, 10:57 PM UTC

djeastm wrote: post #95

gherkinnn wrote: post #35

Earlier quoted context omitted.

To answer your question: talking to a human. Medicine is so much more than "knowledge, experience, and pattern matching", as any patient ever can attest to. Why is it so hard for some people to understand that humans need other humans and human problems can't be solved with technology?

>Medicine is so much more than "knowledge, experience, and pattern matching", as any patient ever can attest to. Humans (doctors/nurses) can still be there to make you feel the warmth of humanity in your darkest times, but if a machine is going to perform better at diagnosing (or perhaps someday performing surgery), then I want the machine. Even now, I'll take a surgeon that's a complete jerk over a nice surgeon any…

I haven't known doctors or nurses to be very warm and fuzzy. I have known them to have real world experience in seeing the outcomes of their actions instead of...

Dude you removed my right thumb I was in for an appendectomy!?

You are so right! I ignored everything you asked for. I am so sorry. I am administering general anesthesia now, then I will prepare you for your next surgery.