OpenAI’s o1 correctly diagnosed 67% of ER patients vs. 50-55% by triage doctors

This is basically the only way how to ethically approach the topic. First you verify performance on “vignettes” as you say. Then if the performance appears satisfying you can continue towards larger tests and more raw sensor modalities. If the results are still promising (both that they statistically agree with the doctors, but also that when they disagree we find the AIs actions to fall benignly). These phases take a lot of time and carefull analysises. And only after that can we carefully design experiments where the AI works together with doctors. For example an experiment where the AI would offer suggestion for next steps to a doctor. These test need to be constructed with great care by teams who are very familiar with medical ethics, statistics and the problems of human decision making. And if the results are still positive just then can we move towards experiments where the humans are supervising the AI less and the AI is more in the driving seat.

Basically to validate this ethically will take decades. So we can’t really fault the researchers that they have only done the first tentative step along this long journey.

> if the Internet somehow goes down at my hospital, the Doctor can still think, while LLM services cannot

Privacy, resiliency and scalability are all best served with local LLMs here.

> If the power goes out at the hospital, the Doctor can still operate, while even local LLMs cannot.

Generators would be the obvious answer there. If we can make machines which outperform human doctors in realworld conditions providing generator backed UPS power for said machines will be a no brainer.

> You're going to need to improve the power efficiency of these models by at least two orders of magnitude before they're generally useful replacements of anything.

Why? Do you have numbers here or just feels?

Particularly given the alarming stories of people being prosecuted for having miscarriages, it feels ridiculous.

If anything I hope more automated diagnostics and triage could help women and POC get better care, but only if there’s safeguards against prejudice. There’s studies showing different rates of pain management across races and sexes, for example. A broken bone is a broken bone, regardless of sex or race.

Quite to the contrary, I think it's extremely trivial to find a task where humans beat LLMs.

For all the money that's been thrown at agentic coding, LLMs still produce substantially worse code than a senior dev. See my own prior comments on this for a concrete example [1].

These trivial failure cases show that there are dimensions to task proficiency - significant ones - that benchmarks fail to capture.

> Is medical diagnosis one of these high judgement tasks?

Situational. I would break diagnosis into three types:

1. The diagnosis comes from objective criteria - laboratory values, vital signs, visual findings, family history. I think LLMs are likely already superior to humans in this case.

2. The diagnosis comes from "chart lore" - reading notes from prior physicians and realizing that there is new context now points to a different diagnosis. (That new context can be the benefit of hindsight into what they already tried and failed and/or new objective data). LLMs do pretty good at this when you point them at datasets where all the prior notes were written by humans, which means that those humans did a nontrivial part of the diagnostic work. What if the prior notes were written by LLMs as well? Will they propagate their own mistakes forward? Yet to be studied in depth.

3. The diagnosis comes from human interaction - knowing the difference between a patient who's high as a bat on crack and one who's delirious from infection; noticing that a patient hesitates slightly before they assure you that they've been taking all their meds as prescribed; etc. I doubt that LLMs will ever beat humans at this, but if LLMs can be proven to be good at point 2, then point 3 alone will not save human physicians.

[1] https://news.ycombinator.com/threads?id=Calavar#47891432

If you, like me, are in the software field, know that this is likely the most comfortable job even invented by humanity, we should really be paid just above the poverty line in exchange.