Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI’s o1 correctly diagnosed 67% of ER patients vs. 50-55% by triage doctors

theguardian.com

Post reply on HN
Print view View original

271–280 of 500 posts

Re: OpenAI’s o1 correctly diagnosed 67% of ER patients vs. 50-55% by triage doctors

#271

Post by skeptic_ai »

gherkinnn wrote: post #35
pixel_popping wrote: post #25

Earlier quoted context omitted.

I agree with you on this specific study, however, I can't really wrap my head about the fact that doctors will be better than AI models on the long-run. After all, medicine is all about knowledge, experience and intelligence (maybe "pattern recognition"), all those, we must assume that the best AI models (especially ones focusing solely in the medical field) would largely beat large majority of humans (aka doctors),…

To answer your question: talking to a human. Medicine is so much more than "knowledge, experience, and pattern matching", as any patient ever can attest to. Why is it so hard for some people to understand that humans need other humans and human problems can't be solved with technology?

You have 2 options

A) nice chatty friendly and cool doctor and can diagnose correctly 50% of the times. B) robotic ai that diagnoses 60% correctly.

What you chose? If you have a disease than can kill your, the ai is 20% more likely to help you and probably prevent. I can’t see too many people choosing human doctor. Anyway I’m sure there will be people that will chose doctor with 10% correctness vs a 100% ai no matter what.

I time is clear there very little human element.

Re: OpenAI’s o1 correctly diagnosed 67% of ER patients vs. 50-55% by triage doctors

#272

Post by parineum »

Neywiny wrote: post #263
squigz wrote: post #212

Earlier quoted context omitted.

I feel like it's possible you misheard/misremember this, considering the temperature for concern is 104.

You are objectively incorrect. A fever is considered 100.4 or 38 C. Here are a few links to prove it: https://my.clevelandclinic.org/health/symptoms/10880-fever https://www.mayoclinic.org/diseases-conditions/fever/symptom... https://www.osfhealthcare.org/blog/whats-considered-a-fever-... https://www.brownhealth.org/be-well/fever-and-body-temperatu... https://www.childrensmercy.org/siteassets/media-documents-fo... I c…

Your not addressing the dispute.

A fever is 38c, great. What the parents said was that you may have misheard because a fever isn't serious until 104. Which is line's up with the language you used.

> and they said it's not a concern until...

Parent is not suggesting that a fever isn't at 100F, they're suggesting that it's not "a concern" until 104F, a number strangely similar to 100.4 that you claim you heard, presumably, while you had a fever.

Re: OpenAI’s o1 correctly diagnosed 67% of ER patients vs. 50-55% by triage doctors

#273

Post by Tenobrus »

o1 has a METR time horizon of around 40 minutes, opus 4.7 has an implied horizon of 18 hours based on its ECI score. this study is on a model that's several generations behind wrt the kind of tasks it can complete. it would be shocking if this number were anywhere near as low with GPT 5.5, to the point it seems nearly totally irrelevant to talk about these results

Re: OpenAI’s o1 correctly diagnosed 67% of ER patients vs. 50-55% by triage doctors

#274

Post by spiralcoaster »

Neywiny wrote: post #263
squigz wrote: post #212

Earlier quoted context omitted.

I feel like it's possible you misheard/misremember this, considering the temperature for concern is 104.

You are objectively incorrect. A fever is considered 100.4 or 38 C. Here are a few links to prove it: https://my.clevelandclinic.org/health/symptoms/10880-fever https://www.mayoclinic.org/diseases-conditions/fever/symptom... https://www.osfhealthcare.org/blog/whats-considered-a-fever-... https://www.brownhealth.org/be-well/fever-and-body-temperatu... https://www.childrensmercy.org/siteassets/media-documents-fo... I c…

Maybe you had trouble re-reading your own comment but I can tell by how you responded here (a cascade of links/references) and a snarky comment ("I can keep going if you'd like") that I'm sure the doctor was glad to be rid of you.

You didn't say the doctor disputed you had a fever. You said the doctor told you the fever wasn't concern until 100.4. Which I'm guessing is your fault for misinterpreting. If you google around, it's very easy to see the fever thresholds.

Here, I'll even paste a summary for you, and I can keep going if you like:

Key Temperature Thresholds

- 100.4°F : The standard definition of a fever.

- 103°F : Contact a healthcare provider

- 104°F : Seek medical attention, particularly if it does not come down with - treatment.

- 105°F : Emergency; seek immediate care.

In one of your own links (clevelandclinic.org), here's an excerpt for you:

When should a fever be treated by a healthcare provider? In adults, fevers less than 103 degrees F (39.4 degrees C) typically aren’t dangerous and aren’t a cause for concern. If your fever rises above that level, make a call to your healthcare provider for treatment.

Re: OpenAI’s o1 correctly diagnosed 67% of ER patients vs. 50-55% by triage doctors

#275

Post by SauntSolaire »

ianbutler wrote: post #240
analog31 wrote: post #161

Earlier quoted context omitted.

I used to think this too. But the past couple of years have soured my taste for "dismantle and replace" of vital institutions. I still think healthcare needs to be reformed, and I hope that insurance will someday be a thing of a past, but I've hung up my chain saw for now.

It's increased mine if it works for the repugnant morons in government right now we can use the same playbook for positive change.

It's easy to destroy but hard to create. If your goal is to further destroy then I suppose that's achievable, but I have a hard time picturing what positive change is going to come from it.

Re: OpenAI’s o1 correctly diagnosed 67% of ER patients vs. 50-55% by triage doctors

#277

Post by manmal »

I know a cardiologist who founded a training & knowledge base startup for doctors. He once told me (that was before LLMs), that it’s super common to tell a patient that the doc needs to look up sthg in their patient history, to then instead google the symptoms. Or, even more often, quickly text a colleague.

I have no way of knowing if this is true. But I‘d rather had a complete, guided prompt be the basis of a diagnosis, than a 2m google search.

Re: OpenAI’s o1 correctly diagnosed 67% of ER patients vs. 50-55% by triage doctors

#278

Post by TurdF3rguson »

n8henrie wrote: post #252
TurdF3rguson wrote: post #245

Earlier quoted context omitted.

> An AI will not pick up on any of that. It will if it trains on data like that. It's all about the training data.

Unfortunately the training data is absolute garbage. Diagnostic standards in (at least emergency, but I think other specialties) medicine are largely a joke -- ultimately it's often either autopsy or "expert consensus." We get to bill more for more serious diagnoses. The amount of patients I see with a "stroke" or "heart attack" diagnosis that clearly had no such thing is truly wild. We can be sued for tens of millio…

You just get 1M doctors to wear body cams for a year. Now you have a model that has thousands of times your experience with patients, encyclopedic knowledge of every ailment including ones that never present in your geography, read all the latest papers, etc..

I don't understand how you think this doesn't win vs a human doctor.

Re: OpenAI’s o1 correctly diagnosed 67% of ER patients vs. 50-55% by triage doctors

#279

Post by [deleted] »

guidedlight wrote: post #270
01100011 wrote: post #134

I wouldn't put much weight in this study, but I think a lot of us can still attest to the usefulness of LLMs in self-diagnostics. The reality in the US is that it is difficult to get the attention and care of a doctor so we're left having to do it ourselves. 10 years ago you'd hear docs complaining about patients coming in with things they found on google but now I don't think there's an alternative. Case in point, I…

I agree. I think the issue with LLM’s are not with the correct diagnoses’s but rather the incorrect ones. Real doctors tend to have a degree of cautiousness. I would rather a real doctor be hesitate and seek more information, than an alarmist LLM suggesting I have cancer.

[deleted]

Re: OpenAI’s o1 correctly diagnosed 67% of ER patients vs. 50-55% by triage doctors

#280

Post by YetAnotherNick »

teleforce wrote: post #265
pixel_popping wrote: post #25

Earlier quoted context omitted.

I agree with you on this specific study, however, I can't really wrap my head about the fact that doctors will be better than AI models on the long-run. After all, medicine is all about knowledge, experience and intelligence (maybe "pattern recognition"), all those, we must assume that the best AI models (especially ones focusing solely in the medical field) would largely beat large majority of humans (aka doctors),…

>What is the specific capability (or combination of capabilities) that people believe will remain permanently (or at least for decades) where a top medical AI cannot match or exceed the performance of a good human doctor? Let's put liability and ethics aside, let's be purely objective about it. You cannot simply put liability and ethics aside, after all there's Hippocatic oath that's fundamental to the practice physi…

Assume if you know for certain that AI has better senstivity and specificity than your local physician for the particular diagnosis, which likely would be the case now or in few years. Would you purposefully get inferior consultation just because of Hippocatic oath?
Post reply on HN