I’ve some family in medicine and it scares me how much they now rely on AI. Some even quote it like Bible.
OpenAI’s o1 correctly diagnosed 67% of ER patients vs. 50-55% by triage doctors
81–90 of 500 posts
Re: OpenAI’s o1 correctly diagnosed 67% of ER patients vs. 50-55% by triage doctors
#82I'd be very very hesitant to trust studies like this. It's very easy to mess up these benchmarks. See for example this recent paper where AI managed to beat radiologists on interpreting x-rays... when the AI didn't even have access to the x-rays: https://arxiv.org/pdf/2603.21687 (on a pre existing "large scale visual question answering benchmark for generalist chest x-ray understanding" that wasn't intentionally mess…
I agree with you on this specific study, however, I can't really wrap my head about the fact that doctors will be better than AI models on the long-run. After all, medicine is all about knowledge, experience and intelligence (maybe "pattern recognition"), all those, we must assume that the best AI models (especially ones focusing solely in the medical field) would largely beat large majority of humans (aka doctors),…
Medicine is about knowledge, but acquiring knowledge may in fact require "breaking out of the box" that AI is increasing behind to avoid touching "touchy subjects" or insulting anyone and so on.
Re: OpenAI’s o1 correctly diagnosed 67% of ER patients vs. 50-55% by triage doctors
#83I'd be very very hesitant to trust studies like this. It's very easy to mess up these benchmarks. See for example this recent paper where AI managed to beat radiologists on interpreting x-rays... when the AI didn't even have access to the x-rays: https://arxiv.org/pdf/2603.21687 (on a pre existing "large scale visual question answering benchmark for generalist chest x-ray understanding" that wasn't intentionally mess…
Weird that this is the case and a new study.
but those kind of x-ray models are already activly used. They are not used though as a only and final diagnosis. Its more like peer review and priorization like check this image first because it seems most critical today.
Re: OpenAI’s o1 correctly diagnosed 67% of ER patients vs. 50-55% by triage doctors
#84I'd be very very hesitant to trust studies like this. It's very easy to mess up these benchmarks. See for example this recent paper where AI managed to beat radiologists on interpreting x-rays... when the AI didn't even have access to the x-rays: https://arxiv.org/pdf/2603.21687 (on a pre existing "large scale visual question answering benchmark for generalist chest x-ray understanding" that wasn't intentionally mess…
I think the bigger takeaway here is that 50% of the time doctors will miss what you have.
I believe in modern medicine but I lost some faith in the American institutions around it when I "diagnosed" my partner with the correct disease that the first rheumatologist dismissed and told them to just stretch. It was officially diagnosed years later, and we lost a lot of time because of it.
Re: OpenAI’s o1 correctly diagnosed 67% of ER patients vs. 50-55% by triage doctors
#85I'd be very very hesitant to trust studies like this. It's very easy to mess up these benchmarks. See for example this recent paper where AI managed to beat radiologists on interpreting x-rays... when the AI didn't even have access to the x-rays: https://arxiv.org/pdf/2603.21687 (on a pre existing "large scale visual question answering benchmark for generalist chest x-ray understanding" that wasn't intentionally mess…
I think the bigger takeaway here is that 50% of the time doctors will miss what you have.
That's not a takeaway here at all.
It's 50% of the time ER doctors working solely from notes, something they never do, in a situation they know is only for a study, will miss what you have.
In real clinical situations the doctors see, hear, smell, and interact with the patients.
Re: OpenAI’s o1 correctly diagnosed 67% of ER patients vs. 50-55% by triage doctors
#86Earlier quoted context omitted.
I agree with you on this specific study, however, I can't really wrap my head about the fact that doctors will be better than AI models on the long-run. After all, medicine is all about knowledge, experience and intelligence (maybe "pattern recognition"), all those, we must assume that the best AI models (especially ones focusing solely in the medical field) would largely beat large majority of humans (aka doctors),…
To answer your question: talking to a human. Medicine is so much more than "knowledge, experience, and pattern matching", as any patient ever can attest to. Why is it so hard for some people to understand that humans need other humans and human problems can't be solved with technology?
One doctor didn't want to give me ritalin, so i went to another one.
One was against it, the other one saw it as a good idea.
I would love to have real data, real statistics etc.
Re: OpenAI’s o1 correctly diagnosed 67% of ER patients vs. 50-55% by triage doctors
#87As a 37 year old male with 2 THRs I'm glad the AI was NOT used in my diagnosis. All the models that I used to look at my x-rays said nothing was wrong, even when adding symptoms. When adding age it said the patient was too young. (I was ~3 months away from wheelchair bound in those x-rays). The worst one was Gemini. Upload an x-ray of just the right hip, and it started to talk about how good the left hip looked like.…
The general AI models are useless if you need precision. They are designed to create/analyze pretty pictures. But specialized models can be inhumanly good. I know, our main product is a model that does _precise_ analysis :)
I'd love to see the output of your system for my x-rays!
Re: OpenAI’s o1 correctly diagnosed 67% of ER patients vs. 50-55% by triage doctors
#88Earlier quoted context omitted.
I agree with you on this specific study, however, I can't really wrap my head about the fact that doctors will be better than AI models on the long-run. After all, medicine is all about knowledge, experience and intelligence (maybe "pattern recognition"), all those, we must assume that the best AI models (especially ones focusing solely in the medical field) would largely beat large majority of humans (aka doctors),…
To answer your question: talking to a human. Medicine is so much more than "knowledge, experience, and pattern matching", as any patient ever can attest to. Why is it so hard for some people to understand that humans need other humans and human problems can't be solved with technology?
> human problems can't be solved with technology
How are you defining technology? How are you defining human problems? Inventions are created to solve human problems, not theoretical problems of fictional universe. Do X-rays, refrigerators, phones and even looms solve problems for nonhumans?
Claiming something that sounds deep doesn’t make it an axiom.
Re: OpenAI’s o1 correctly diagnosed 67% of ER patients vs. 50-55% by triage doctors
#89Earlier quoted context omitted.
The general AI models are useless if you need precision. They are designed to create/analyze pretty pictures. But specialized models can be inhumanly good. I know, our main product is a model that does _precise_ analysis :)
I'd love to see the output of your system for my x-rays!
Sorry, it's on the entirely wrong side of the spectrum. We're doing geospatial analysis. Although it'd be hilarious to see what it thinks about X-Rays.
Re: OpenAI’s o1 correctly diagnosed 67% of ER patients vs. 50-55% by triage doctors
#90I'd be very very hesitant to trust studies like this. It's very easy to mess up these benchmarks. See for example this recent paper where AI managed to beat radiologists on interpreting x-rays... when the AI didn't even have access to the x-rays: https://arxiv.org/pdf/2603.21687 (on a pre existing "large scale visual question answering benchmark for generalist chest x-ray understanding" that wasn't intentionally mess…
I think AI can be useful in any kind of context interpretation, but not make a decision. Could be running in the background on patient data and message the doctor "I see X in the diagnostic, have you ruled out Y, as it fits for reasons a, b, c?" I like my coding agents the same way, inform me during review on things that I've missed. Instead of having me comb through what it generates on a first pass.
[flagged]