Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI’s o1 correctly diagnosed 67% of ER patients vs. 50-55% by triage doctors

theguardian.com

Post reply on HN
Print view View original

301–310 of 500 posts

Re: OpenAI’s o1 correctly diagnosed 67% of ER patients vs. 50-55% by triage doctors

#301

Post by davycro »

TurdF3rguson wrote: post #278
n8henrie wrote: post #252

Earlier quoted context omitted.

Unfortunately the training data is absolute garbage. Diagnostic standards in (at least emergency, but I think other specialties) medicine are largely a joke -- ultimately it's often either autopsy or "expert consensus." We get to bill more for more serious diagnoses. The amount of patients I see with a "stroke" or "heart attack" diagnosis that clearly had no such thing is truly wild. We can be sued for tens of millio…

You just get 1M doctors to wear body cams for a year. Now you have a model that has thousands of times your experience with patients, encyclopedic knowledge of every ailment including ones that never present in your geography, read all the latest papers, etc.. I don't understand how you think this doesn't win vs a human doctor.

This wouldn't solve the problem of diagnostic standards. Let's say you are a pediatrician and want to predict which kids with bronchiolitis will develop respiratory failure and need the ICU versus the ones who can go home. How do you determine from the body cams which kids had bronchiolitis in the first place? Bronchiolitis is a clinical diagnosis with symptoms that overlap with other respiratory illnesses such as asthma, bacterial pneumonia, croup, foreign body ingestion, etc.

Re: OpenAI’s o1 correctly diagnosed 67% of ER patients vs. 50-55% by triage doctors

#302

Post by theshrike79 »

troupo wrote: post #65
theshrike79 wrote: post #28

I'll repeat my idea on how this MUST be done: 1. AI gets data about the patient and makes a diagnosis. This is NOT shown to doctor yet. 2. Doctor does their stuff, writes down their diagnosis. This diagnosis is locked down and versioned. 3. Doctor sees AI's diagnosis 4. Doctor can adjust their diagnosis, BUT the original stays in the system. This way the AI stays as the assistant and won't affect the doctor's decisio…

5. Doctors delegate everything to AI assistants because humans are lazy, especially if those AI assistants are correct some significant portion of the time

Step 2 prevents that. It's not there by accident.

They need to write down their (initial) diagnosis before the AI answer is shown.

Re: OpenAI’s o1 correctly diagnosed 67% of ER patients vs. 50-55% by triage doctors

#303

Post by SirHumphrey »

DrewADesign wrote: post #228
hyperpape wrote: post #200

Earlier quoted context omitted.

> we must assume that the best AI models (especially ones focusing solely in the medical field) would largely beat large majority of humans (aka doctors), if we already have this assumption for software engineers, we should have it for this field as well, This is a pretty wild leap. Code has a lot of hooks for training via hill-climbing during post-training. During post-training, you can literally set up arbitrary sc…

Code is pretty much the perfect use case for LLMs… text-based, very pattern-oriented, extremely limited complexity compared to biological systems, etc. I suspect even prose is largely considered acceptable in professional uses because we haven’t developed a sensitivity to the artifice, and we probably won’t catch up to the LLMs in that arms race for a bit. However, we always manage to develop a distaste for cheap imi…

And with the code, the closer you come to the physical world the worse LLMs fair.

Claude can’t really write Openscad and when I was debugging some map projections code last week it struggled a lot more than usual.

Re: OpenAI’s o1 correctly diagnosed 67% of ER patients vs. 50-55% by triage doctors

#304

Post by thaumasiotes »

kaikai wrote: post #184
thaumasiotes wrote: post #151

Earlier quoted context omitted.

> My daughter is in her 20s now and is still small -- it's just the way she is. When she goes to see her primary, do you know what their first question is? "When was your last period." Is that supposed to be a problem? How does it connect to the story in your comment? The question seems to be warranted to me, since being underweight can stop you from menstruating. So if you find someone thin and her last period was o…

I’m a normal weight, and get asked the same question. More importantly, I can tell them, “I have a regular cycle” and they WILL NOT take that as an answer. I HAVE to give them a date, and they will ask me to make one up if I can’t remember or want to decline giving them that information. Particularly given the alarming stories of people being prosecuted for having miscarriages, it feels ridiculous. If anything I hope…

> and they will ask me to make one up if I can’t remember or want to decline giving them that information

Doesn't this suggest that they don't care what the answer is?

Re: OpenAI’s o1 correctly diagnosed 67% of ER patients vs. 50-55% by triage doctors

#306

Post by tossandthrow »

creativeSlumber wrote: post #36

> "An AI and a pair of human doctors were each given the same standard electronic health record to read" This is handicapping the human doctors abilities. There is a lot more information a human doctor can gather even with a brief observation of the patient.

You could say the same about the Ai. Ai is incredibly well suited for extracting knowledge through chats.

In this regard. A doctor also just have 15 minutes for an interview. An Ai can be with the patient for days leading up to a consultation.

So if we remove this "handicap" this Ai will likely really start to win.

Re: OpenAI’s o1 correctly diagnosed 67% of ER patients vs. 50-55% by triage doctors

#307

Post by dolkycape »

SkiFire13 wrote: post #296
pixel_popping wrote: post #25

Earlier quoted context omitted.

I agree with you on this specific study, however, I can't really wrap my head about the fact that doctors will be better than AI models on the long-run. After all, medicine is all about knowledge, experience and intelligence (maybe "pattern recognition"), all those, we must assume that the best AI models (especially ones focusing solely in the medical field) would largely beat large majority of humans (aka doctors),…

> we must assume that the best AI models (especially ones focusing solely in the medical field) would largely beat large majority of humans (aka doctors), if we already have this assumption for software engineers You first have to assume this for software engineers. Not everyone agree with that (note: that doesn't mean the same people don't agree that AI is not _useful_). AIs still have a ton of issues that would be…

Doctors do that all the time though. That's why drugs are dispensed by a pharmacist who double checks it.

Re: OpenAI’s o1 correctly diagnosed 67% of ER patients vs. 50-55% by triage doctors

#308

Post by ianbutler »

SauntSolaire wrote: post #275
ianbutler wrote: post #240

Earlier quoted context omitted.

It's increased mine if it works for the repugnant morons in government right now we can use the same playbook for positive change.

It's easy to destroy but hard to create. If your goal is to further destroy then I suppose that's achievable, but I have a hard time picturing what positive change is going to come from it.

No offense, but this comes off as passive indifference and while I've heard people say things like this all my life it has broadly resulted in watching 30 years of societal decay. I can't help but think this is wrong.

We should have stacked the courts ourselves, brandished executive orders etc, had some spine.

Edit: I think I need to make clear my thinking that the right has selectively destroyed institutions and levied them in other areas where it makes sense for their agenda. It's not been wanton. So when I say leverage the playbook it's not a one sided act of destruction.

Re: OpenAI’s o1 correctly diagnosed 67% of ER patients vs. 50-55% by triage doctors

#309

Post by rrr_oh_man »

thaumasiotes wrote: post #304
kaikai wrote: post #184

Earlier quoted context omitted.

I’m a normal weight, and get asked the same question. More importantly, I can tell them, “I have a regular cycle” and they WILL NOT take that as an answer. I HAVE to give them a date, and they will ask me to make one up if I can’t remember or want to decline giving them that information. Particularly given the alarming stories of people being prosecuted for having miscarriages, it feels ridiculous. If anything I hope…

> and they will ask me to make one up if I can’t remember or want to decline giving them that information Doesn't this suggest that they don't care what the answer is?

It sounds like a form to be filled out…

Re: OpenAI’s o1 correctly diagnosed 67% of ER patients vs. 50-55% by triage doctors

#310

Post by rrr_oh_man »

phoronixrly wrote: post #208
calmworm wrote: post #204

Earlier quoted context omitted.

You could manipulate or write the input/prompt in a way that would make it recommend any drug you wanted.

You think that in the country of the war on drugs such a thing will be approved?

They already approve / tolerate offshore call center doctors
Post reply on HN