[flagged]
OpenAI’s o1 correctly diagnosed 67% of ER patients vs. 50-55% by triage doctors
101–110 of 500 posts
Re: OpenAI’s o1 correctly diagnosed 67% of ER patients vs. 50-55% by triage doctors
#102This is a rather new article about an old model...
Study design, data collection, analysis, and peer review take time. O1 came out a little over 1.5 years ago
At this point the study is already mostly irrelevant because the model in question has long been far surpassed by new models. It seems traditional publishing doesn't work for really fast moving fields.
Re: OpenAI’s o1 correctly diagnosed 67% of ER patients vs. 50-55% by triage doctors
#103Earlier quoted context omitted.
To answer your question: talking to a human. Medicine is so much more than "knowledge, experience, and pattern matching", as any patient ever can attest to. Why is it so hard for some people to understand that humans need other humans and human problems can't be solved with technology?
>Medicine is so much more than "knowledge, experience, and pattern matching", as any patient ever can attest to. Humans (doctors/nurses) can still be there to make you feel the warmth of humanity in your darkest times, but if a machine is going to perform better at diagnosing (or perhaps someday performing surgery), then I want the machine. Even now, I'll take a surgeon that's a complete jerk over a nice surgeon any…
I'd be a little bit careful here - being a jerk is quite different to non-conformity / red sneaker effect in surgery and it is not a quality you should look for.
The truly compassionate surgeons will want to improve their skills because they care about their patients. They care if they develop complications and may feel terrible if they do, the jerk may not. Being a jerk may mean that the surgeon can rise to the top, but it may not be due to surgical skill at all, they may be better at navigating politics etc.
Re: OpenAI’s o1 correctly diagnosed 67% of ER patients vs. 50-55% by triage doctors
#104I’ve had much better luck with diagnosis of my own family’s issues than with doctors. Usually now, I’m feeding them more information to begin with, so that their 30 minute office visits are not wasted, requiring another expensive follow up appointment. While I’m sure there can be ways in which such studies are wrong, it’s very obvious that AI can accelerate work in many of these areas where we seek out professional h…
It can speed up some aspects of work, but please don't trust some llm with variable quality of output more than professional. If you don't like current doctor try another, most are in the business of helping other people. If you have string of issues with 10 last doctors though, then issue is, most probably, you... My wife is a GP, and easily 1/3 of her patients have also some minor-but-visible mental issue. 1-2 out…
Respectfully, as someone with a family with plenty of medical issues and having experienced plenty of useless doctors, the onus is now on medical professionals to prove their worth. They are a second option and most of their remaining value is in the license to prescribe medication, after being told by laymen what medication is appropriate. They're using the same tools I am and they're worse at evaluating them.
Doctors thinking patients are arrogant is an age old problem.
Re: OpenAI’s o1 correctly diagnosed 67% of ER patients vs. 50-55% by triage doctors
#105I advise a medical non profit and we ran a series of tests against cases doctors input to our system looking for specialist recommendations.
Our findings found that gpt-5-mini performed better than gpt-5, sonnet 4 and medgemma.
I think these studies are very hard to accurately score. But in any case, AI seems to do a very good job compared to humans. Unsurprising, really.
Re: OpenAI’s o1 correctly diagnosed 67% of ER patients vs. 50-55% by triage doctors
#106I'd be very very hesitant to trust studies like this. It's very easy to mess up these benchmarks. See for example this recent paper where AI managed to beat radiologists on interpreting x-rays... when the AI didn't even have access to the x-rays: https://arxiv.org/pdf/2603.21687 (on a pre existing "large scale visual question answering benchmark for generalist chest x-ray understanding" that wasn't intentionally mess…
I think it's plausible since doctors tend to have human cognitive biases and miss things. People tend to fixate on patterns they're most familiar with.
Re: OpenAI’s o1 correctly diagnosed 67% of ER patients vs. 50-55% by triage doctors
#107Earlier quoted context omitted.
To answer your question: talking to a human. Medicine is so much more than "knowledge, experience, and pattern matching", as any patient ever can attest to. Why is it so hard for some people to understand that humans need other humans and human problems can't be solved with technology?
If you read the study, the whole conclusion is much less spectacular than the article. What the article really pushes happened: patients -> AI -> diagnosis (you know, with a camera, or perhaps a telephone I guess) What REALLY happened patients -> nurse/MD -> text description of symptoms -> MD -> question (as in MD asked a relevant diagnostic question, such as "is this the result of a lung infection?", or "what lab te…
This. The fact that the ai projects have to spin so hard should be tipping people off. But for some reason it doesn’t.
Re: OpenAI’s o1 correctly diagnosed 67% of ER patients vs. 50-55% by triage doctors
#108radiology already had its "AI beats doctors" moment. radiologists are still here. what changed first was the workflow, not the specialty. er is probably next.
I don't think radiology has had that moment at all. Computer programming is much closer, if not, at that moment right now.
Re: OpenAI’s o1 correctly diagnosed 67% of ER patients vs. 50-55% by triage doctors
#109I wonder about the nuance within the data. Like does AI do much worse with children than adults, but still better overall for example. Or biological male vs female. I think we'd want it to do better across all groups, ages etc so we're not introducing some kind of horrible bias resulting in deaths or serious health consequences for some groups
Re: OpenAI’s o1 correctly diagnosed 67% of ER patients vs. 50-55% by triage doctors
#110I'd be very very hesitant to trust studies like this. It's very easy to mess up these benchmarks. See for example this recent paper where AI managed to beat radiologists on interpreting x-rays... when the AI didn't even have access to the x-rays: https://arxiv.org/pdf/2603.21687 (on a pre existing "large scale visual question answering benchmark for generalist chest x-ray understanding" that wasn't intentionally mess…
I think it's plausible since doctors tend to have human cognitive biases and miss things. People tend to fixate on patterns they're most familiar with.
A bold claim to suggest that LLMs aren’t prone to biases of their own which are less understood.