Live data from Hacker News

OpenAI’s o1 correctly diagnosed 67% of ER patients vs. 50-55% by triage doctors

theguardian.com

171–180 of 500 posts

Re: OpenAI’s o1 correctly diagnosed 67% of ER patients vs. 50-55% by triage doctors

#171
post #144
post #142

Earlier quoted context omitted.

You’re holding on to the intuition (hope) that we are smarter than the LLMs in some hard to define way. Maybe. But it’s getting harder and harder to define a task that humans beat LLMs on. On pretty much any easily quantifiable test of knowledge or reasoning, the machines win. I agree experienced humans are still better on “judgement” tasks in their field. But the judgement tasks are kinda necessarily ones where ther…

LLM’s operate on a mechanical form of intelligence one that at present is not adaptive to changes in the environment. If the latter part of your post were true, how come the demand for radiologists has grown? The problem with this place is it’s full of people who don’t understand nuance. And your post demonstrates this emphatically.

For me there are a few main takeaways on how AI _could_ supersede the average ER doctor.

The first is that a technical solution can be trained on _ALL_ medical data and have access to it all in the moment. It is difficult to assume a doctor could also achieve this.

The second is that for medical cases understanding the sum of all symptoms and the patients vitals would lead to an accurate diagnosis a majority of the time. AI/ML is entirely about pattern recognition, when you combine this with point one, you end up with a system that can quickly diagnose a large portion of patients in extremely short timeframes.

On a different note, I think we can leave the ad-hominem attacks at home please.

Re: OpenAI’s o1 correctly diagnosed 67% of ER patients vs. 50-55% by triage doctors

#172

Earlier quoted context omitted.

I agree with you on this specific study, however, I can't really wrap my head about the fact that doctors will be better than AI models on the long-run. After all, medicine is all about knowledge, experience and intelligence (maybe "pattern recognition"), all those, we must assume that the best AI models (especially ones focusing solely in the medical field) would largely beat large majority of humans (aka doctors),…

To answer your question: talking to a human. Medicine is so much more than "knowledge, experience, and pattern matching", as any patient ever can attest to. Why is it so hard for some people to understand that humans need other humans and human problems can't be solved with technology?

Yes talking to a human is good and necessary. But for diagnostics humans are not good at it. I'm happy for to human to use a tricorder and then tell me the answer.

Re: OpenAI’s o1 correctly diagnosed 67% of ER patients vs. 50-55% by triage doctors

#173
post #92

Earlier quoted context omitted.

If you read the study, the whole conclusion is much less spectacular than the article. What the article really pushes happened: patients -> AI -> diagnosis (you know, with a camera, or perhaps a telephone I guess) What REALLY happened patients -> nurse/MD -> text description of symptoms -> MD -> question (as in MD asked a relevant diagnostic question, such as "is this the result of a lung infection?", or "what lab te…

This. The fact that the ai projects have to spin so hard should be tipping people off. But for some reason it doesn’t.

People only read headlines and offload their critical thinking skills to the companies who are selling them in their next publication. It's sad.

Re: OpenAI’s o1 correctly diagnosed 67% of ER patients vs. 50-55% by triage doctors

#174
This reminds me GPT-4 era studies where the LLM was better in a Law school exam than a student. We are not in 2023 anymore, or in the case of medicine, are we? If yes, this is bad news for health related applications as the low hanging fruits in LLM have been cut off.

Re: OpenAI’s o1 correctly diagnosed 67% of ER patients vs. 50-55% by triage doctors

#175

Earlier quoted context omitted.

I agree with you on this specific study, however, I can't really wrap my head about the fact that doctors will be better than AI models on the long-run. After all, medicine is all about knowledge, experience and intelligence (maybe "pattern recognition"), all those, we must assume that the best AI models (especially ones focusing solely in the medical field) would largely beat large majority of humans (aka doctors),…

Last time I went to the ER the doctor used a scope to look down my throat and check everything seemed fine. I don't think pure AI like ChatGPT will be able to do that any time soon. Maybe a medical robot with AI will one day, but that seems at least a few years off.

Yes I don't want a robot shoving anything down my throat anytime soon. I don't even want my car connected to the Internet. Whatever happened to people who kept a loaded handgun in case their printer acted up?

Re: OpenAI’s o1 correctly diagnosed 67% of ER patients vs. 50-55% by triage doctors

#176
post #38

Earlier quoted context omitted.

I agree with you on this specific study, however, I can't really wrap my head about the fact that doctors will be better than AI models on the long-run. After all, medicine is all about knowledge, experience and intelligence (maybe "pattern recognition"), all those, we must assume that the best AI models (especially ones focusing solely in the medical field) would largely beat large majority of humans (aka doctors),…

But liability and ethics cannot be put aside. If treatments were free of cost and perfectly address problems, then a correct diagnosis would always lead to the optimal patient outcome. In that scenario, AI diagnosis will be like code generation and go asymptotic to perfection as models improve. But a doctor's job in the real world today is to navigate a total mess of uncertainty: about the expected outcome of treatme…

>AI diagnosis will be like code generation and go asymptotic to perfection as models improve

uhhhhhhh, I'm pretty behind-the-times on this stuff so I could be the one who's wrong here but I don't believe that has happened????

But anyways that nitpicking aside I agree with you wholeheartedly that reducing the doctor's job to diagnosis (and specifically whatever subset of that can be done by a machine-learning model that doesn't even get to physically interact with the patient) is extremely myopic and probably a bit insulting towards actual doctors.

Re: OpenAI’s o1 correctly diagnosed 67% of ER patients vs. 50-55% by triage doctors

#177
post #68

Believable and not shocking. LLMs literally may have saved my sons and potentially her mother too by allowing us to fact check a lot of non sense data and scare tactics by a group of at least 5 different doctors ambushing us to make a life changing decision in minutes. The problem is doctors, at least in the US, prioritize liability exposure over patients long term outcomes. Let’s say you need an intervention where t…

Is the group of at least 5 different doctors ambushing you, in the room with us right now? Was it 5, or more like 15, or 50? Would it have been more or less frightening if it was a group of the same doctor, but like 40 of him?

I don’t know if you’ve just never had bad healthcare, but this story does not seem unbelievable to me.

I’ve had doctors try to convince me not to pursue medical care, that problems of people close to me were not real and purely psychological, and I’ve personally required emergency surgery due to inaction. In every case there were obvious signs and symptoms.

Doctors are not good at their jobs. In the US, we’ve done a particularly stupid combination of forcing them to incur legal liability and intermediating everything with insurance, both of which impact the care people actually receive.

Re: OpenAI’s o1 correctly diagnosed 67% of ER patients vs. 50-55% by triage doctors

#178

Earlier quoted context omitted.

To answer your question: talking to a human. Medicine is so much more than "knowledge, experience, and pattern matching", as any patient ever can attest to. Why is it so hard for some people to understand that humans need other humans and human problems can't be solved with technology?

One doctor didn't want to give me ritalin, so i went to another one. One was against it, the other one saw it as a good idea. I would love to have real data, real statistics etc.

[flagged]

Re: OpenAI’s o1 correctly diagnosed 67% of ER patients vs. 50-55% by triage doctors

#179

Earlier quoted context omitted.

I agree with you on this specific study, however, I can't really wrap my head about the fact that doctors will be better than AI models on the long-run. After all, medicine is all about knowledge, experience and intelligence (maybe "pattern recognition"), all those, we must assume that the best AI models (especially ones focusing solely in the medical field) would largely beat large majority of humans (aka doctors),…

To answer your question: talking to a human. Medicine is so much more than "knowledge, experience, and pattern matching", as any patient ever can attest to. Why is it so hard for some people to understand that humans need other humans and human problems can't be solved with technology?

LLMs are a distillation of human.

Re: OpenAI’s o1 correctly diagnosed 67% of ER patients vs. 50-55% by triage doctors

#180

The negative reactions here are baffling me. The fact that we can even get to say 30% with computer is amazing. So much hatred towards AI and anything from the frontier labs like OpenAI (or Goog for that matter) makes no sense.

I for one am delighted for my acquaintances in the medical field with their cushy, cartel-supported salaries to feel the existential dread of AI coming for their jobs like I have

I'm sorry that you are feeling existential dread about your career. It could help to stop listening to the hype that the people selling AI are spewing and take a hard look at the tools themselves. Like most products, they aren't as good as the salespeople say they are. Also, take any predictions for how these products will do in the future with a huge grain of salt. Predicting the future is very difficult. It's taken us 70 years of computer and AI research and development to get to this point. It's likely that the rate of improvement will not change drastically. Yes, things are changing, but the singularity (still) is not coming tomorrow
Post reply on HN