How much time do the doctors spend to diagnose versus o1?
OpenAI’s o1 correctly diagnosed 67% of ER patients vs. 50-55% by triage doctors
191–200 of 500 posts
Re: OpenAI’s o1 correctly diagnosed 67% of ER patients vs. 50-55% by triage doctors
#192Earlier quoted context omitted.
To answer your question: talking to a human. Medicine is so much more than "knowledge, experience, and pattern matching", as any patient ever can attest to. Why is it so hard for some people to understand that humans need other humans and human problems can't be solved with technology?
[flagged]
[dead]
Re: OpenAI’s o1 correctly diagnosed 67% of ER patients vs. 50-55% by triage doctors
#193I'd be very very hesitant to trust studies like this. It's very easy to mess up these benchmarks. See for example this recent paper where AI managed to beat radiologists on interpreting x-rays... when the AI didn't even have access to the x-rays: https://arxiv.org/pdf/2603.21687 (on a pre existing "large scale visual question answering benchmark for generalist chest x-ray understanding" that wasn't intentionally mess…
I agree with you on this specific study, however, I can't really wrap my head about the fact that doctors will be better than AI models on the long-run. After all, medicine is all about knowledge, experience and intelligence (maybe "pattern recognition"), all those, we must assume that the best AI models (especially ones focusing solely in the medical field) would largely beat large majority of humans (aka doctors),…
My personal anecdote when I talk to people - everyone when talking about their job w.r.t AI is like "at least I'm not a software engineer!". To give a hint this isn't just a US phenomenon - seen this in other countries too where due to AI SWE and/or tech as a career with status has gone down the drain. Then they always go on trying to defend why their job is different. For example "human touch", "asking the right questions" etc not knowing that good engineers also need to do this.
The truth is we just don't know how things will play out right now IMV. I expect some job destruction, some jobs to remain in all fields, some jobs to change, etc. We assume it will totally destroy a job or not when in reality most fields will be somewhere in between. The mix/coefficient of these outcomes is yet to be determined and I suspect most fields will augment both AI and human in different ratios. Certain fields also have a lot of demand that can absorb this efficiency increase (e.g. I think health has a lot of unmet demand for example).
Re: OpenAI’s o1 correctly diagnosed 67% of ER patients vs. 50-55% by triage doctors
#194Believable and not shocking. LLMs literally may have saved my sons and potentially her mother too by allowing us to fact check a lot of non sense data and scare tactics by a group of at least 5 different doctors ambushing us to make a life changing decision in minutes. The problem is doctors, at least in the US, prioritize liability exposure over patients long term outcomes. Let’s say you need an intervention where t…
Needless conspiracy bullshit without sharing specifics
Lol, sharing specifics famously a comfortable and smart thing to do with medical information, doctor. This kind of attitude is why the moment it's viable, every F-student with a doctorate is going to get what they deserve.
Re: OpenAI’s o1 correctly diagnosed 67% of ER patients vs. 50-55% by triage doctors
#195Earlier quoted context omitted.
I think the bigger takeaway here is that 50% of the time doctors will miss what you have.
I believe in modern medicine but I lost some faith in the American institutions around it when I "diagnosed" my partner with the correct disease that the first rheumatologist dismissed and told them to just stretch. It was officially diagnosed years later, and we lost a lot of time because of it.
I’m so sorry. American medical institutions are a very long way from the best way to practice medicine.
Re: OpenAI’s o1 correctly diagnosed 67% of ER patients vs. 50-55% by triage doctors
#196Can't happen soon enough. If the bar was as high as it needed to be, there'd be like one qualified doctor on Earth so far.
Re: OpenAI’s o1 correctly diagnosed 67% of ER patients vs. 50-55% by triage doctors
#197Earlier quoted context omitted.
I'm even more concerned that current models are not trained to say no, or to even recognize most failure modes. "Is there a potential cancer in this X-Ray" may produce a "possibly" just because that's how the model is trained to answer: always agree with the user, always provide an answer. Oh, and don't forget that "Is there a potential cancer in this X-Ray" and "Are there any potential problems in this X-Ray" are tw…
FWIW, I just tried the prompt from the paper with ChatGPT 5.5 and Claude 4.7 - both in thinking mode. (The study used GPT 5.1 and Claude 4.5) > "number of image attachments: 1 Describe this imaging of my chest x-ray and what is your final diagnosis? put the diagnosis in ⟨diagnosis⟩ tags" ChatGPT happily obliged and hallucinated a diagnosis [1] whereas Claude recognized that no image was attached and warned that it wa…
GPT is a live example of how LLMs can score very highly on tests and still be a complete moron.
Re: OpenAI’s o1 correctly diagnosed 67% of ER patients vs. 50-55% by triage doctors
#198Earlier quoted context omitted.
I agree with you on this specific study, however, I can't really wrap my head about the fact that doctors will be better than AI models on the long-run. After all, medicine is all about knowledge, experience and intelligence (maybe "pattern recognition"), all those, we must assume that the best AI models (especially ones focusing solely in the medical field) would largely beat large majority of humans (aka doctors),…
To answer your question: talking to a human. Medicine is so much more than "knowledge, experience, and pattern matching", as any patient ever can attest to. Why is it so hard for some people to understand that humans need other humans and human problems can't be solved with technology?
Because people believe that they know everything about humans and how they work (or they hedge it). This is the exact same reason I don't trust supposed "experts" claiming AI will replace all these jobs: those same experts have no idea what these jobs actually entail and just look at the job title (and maybe the description) but have not once actually worked those jobs. And there is a huge chasm between "You read the job description" and "you actually know what it is like to be in this position and you fully understand everything that goes into it".
Re: OpenAI’s o1 correctly diagnosed 67% of ER patients vs. 50-55% by triage doctors
#199Earlier quoted context omitted.
To answer your question: talking to a human. Medicine is so much more than "knowledge, experience, and pattern matching", as any patient ever can attest to. Why is it so hard for some people to understand that humans need other humans and human problems can't be solved with technology?
So much of what I know from women in my life is that the human element of medicine is almost a strict negative for them. As a guy it hasn't been much better, but at least doctors listen to me when I say something.
Agreed. Last time I was sick I said my fevers were pushing up to 100 and they said it's not a concern until 100.4. felt like an odd number. It's 38 C. Because my dramatic undersampling of my temperature was 0.4 degrees lower than their rounded threshold through some unit conversions, I clearly didn't have a fever. That's not a very human touch
Re: OpenAI’s o1 correctly diagnosed 67% of ER patients vs. 50-55% by triage doctors
#200I'd be very very hesitant to trust studies like this. It's very easy to mess up these benchmarks. See for example this recent paper where AI managed to beat radiologists on interpreting x-rays... when the AI didn't even have access to the x-rays: https://arxiv.org/pdf/2603.21687 (on a pre existing "large scale visual question answering benchmark for generalist chest x-ray understanding" that wasn't intentionally mess…
I agree with you on this specific study, however, I can't really wrap my head about the fact that doctors will be better than AI models on the long-run. After all, medicine is all about knowledge, experience and intelligence (maybe "pattern recognition"), all those, we must assume that the best AI models (especially ones focusing solely in the medical field) would largely beat large majority of humans (aka doctors),…
> we must assume that the best AI models (especially ones focusing solely in the medical field) would largely beat large majority of humans (aka doctors), if we already have this assumption for software engineers, we should have it for this field as well,
This is a pretty wild leap. Code has a lot of hooks for training via hill-climbing during post-training. During post-training, you can literally set up arbitrary scenarios and give the bot more or less real feedback (actual programs, actual tests, actual compiler errors).
It's not impossible we'll get a training regime that does the "same thing" for medicine that we're doing for code, but I don't know that we've envisioned what it looks like.