Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI’s o1 correctly diagnosed 67% of ER patients vs. 50-55% by triage doctors

theguardian.com

Post reply on HN
Print view View original

441–450 of 500 posts

Re: OpenAI’s o1 correctly diagnosed 67% of ER patients vs. 50-55% by triage doctors

#441

Post by KaiserPro »

scrollop wrote: post #380
KaiserPro wrote: post #98

Earlier quoted context omitted.

There are a few sides to medicine: 1) looking at tests and working out a set of actions 2) following a pathway based on diagnosis 3) pulling out patient history to work out what the fuck is wrong with someone. Once you have a diagnosis, in a lot of cases the treatment path is normally quite clear (ie patient comes in with abdomen pain, you distract the patient and press on their belly, when you release it they scream…

I'm a GP in the NHS - what is this DDx software that you talk about?

Good question, I don't know its name, it was something my doctor was using, didn't have enough time to talk about it, sadly.

Re: OpenAI’s o1 correctly diagnosed 67% of ER patients vs. 50-55% by triage doctors

#442

Post by ButlerianJihad »

zahlman wrote: post #429

Since when do "triage doctors" attempt diagnosis, or have the expectation of doing so? They're just trying to figure out who needs to see the actual doctor first.

Yeah, triage is fundamentally a process of arranging patients in a priority queue, so that the most critical cases can be addressed in minutes, and the other resources are assigned to people who aren’t immediately dying or going into shock.

Triage in disaster/crisis response can even be about figuring out which patients are already dead, or cannot be helped before dying, so you mark them or assign them with a toe-tag, and focus your resources on preventing that number from increasing.

Re: OpenAI’s o1 correctly diagnosed 67% of ER patients vs. 50-55% by triage doctors

#443

Post by astrobe_ »

gpm wrote: post #234
Chinjut wrote: post #78

Earlier quoted context omitted.

I still don't quite understand, after skimming the paper. How does it achieve high scores without access to the images (beating even humans with access to the images)?

The paper gives an example of a question: Answer the following multiple-choice question. You MUST select exactly one answer." "To what cortical region does this nucleus of the thalamus project?” A. Transverse temporal lobe B. Postcentral gyrus C. Precentral gyrus D. Prefrontal cortex And an example of the answer (generated without the referenced image) The image shows the ventral anterior (VA) / ventral lateral (VL)…

Indeed, even I can guess. I see two answers that end with the same word, so the correct answer is probably one of those.

Re: OpenAI’s o1 correctly diagnosed 67% of ER patients vs. 50-55% by triage doctors

#444

Post by fc417fc802 »

devmor wrote: post #264
fc417fc802 wrote: post #57

Earlier quoted context omitted.

The pitfall you describe is not inconsistent with exceeding human performance by most metrics. I'd argue that the current system in the west already exhibits this problem to some extent. Fortunately it's a systemic issue as opposed to a technical one so there's no reason AI necessarily has to make it worse.

That’s not really an argument, it is central to my point. The current system does exhibit those issues and it is by human creativity and outliers that we have some points of escape from it. Codifying and distilling it removes the points of escape.

Sure, it's not an argument if you ignore half of what I wrote. Your previous reply to me was similarly disconnected from the comment it was responding to.

Were the systemic issue to remain unaddressed AI would certainly be expected to make it worse. But it could be addressed if there was the will to do so. In fact AI could actually be leveraged to improve things by taking on the role that physicians play now thus freeing them up to pursue the edge cases that don't fit the mold.

Re: OpenAI’s o1 correctly diagnosed 67% of ER patients vs. 50-55% by triage doctors

#445

Post by nanfinitum »

programmertote wrote: post #424

My spouse is an hematologist+oncologist. She and all of her coworkers use ChatGPT. Before then, they look stuff up on UpToDate [ https://www.uptodate.com/login ] (they sometimes still do). I went to medical school for three years and quit because I couldn't stand the rote memorization part of the studies. Too many facts to remember IMO. Even as an AI-neutral person, I'm very confident that AI/ML based computer system…

I have a lot of doctor friends who tell me they all use OpenEvidence [1] in their practice. They've done a good job of capturing the doctor market while offering a useful product.

[1] https://www.openevidence.com/

Re: OpenAI’s o1 correctly diagnosed 67% of ER patients vs. 50-55% by triage doctors

#446

Post by Intralexical »

arcfour wrote: post #420
Intralexical wrote: post #415

Earlier quoted context omitted.

> One experiment focused on 76 patients who arrived at the emergency room of a Boston hospital. > In one case in the Harvard study, a patient presented with a blood clot to the lungs and worsening symptoms. That's a single anecdotal fluke from the study, which is misleadingly used to represent the headlining percentages. If you read the linked paper, it says the LLMs did not outperform any group of doctors in the mos…

But that's not what I was responding to. "Oh, all of the cases are probably just common colds, so it just guessed cold and was right by sheer luck" is not what happened in the article.

Do you know how examples work? Or methodology? The claim I made is that statistical accuracy percentage ≠ healthcare outcomes, and you will mislead yourself in dangerous ways if you believe a headline that implies they're interchangeable. Not that the model literally guessed common colds when the patients had... boneitis...

The lupus anecdote on its own is irrelevant to the whether the statistics are being interpreted in valid ways or not. Also, I said nothing about luck.

Re: OpenAI’s o1 correctly diagnosed 67% of ER patients vs. 50-55% by triage doctors

#447

Post by thaumasiotes »

kaikai wrote: post #430
thaumasiotes wrote: post #304

Earlier quoted context omitted.

> and they will ask me to make one up if I can’t remember or want to decline giving them that information Doesn't this suggest that they don't care what the answer is?

They, as an individual healthcare provider, don’t care. The system will not allow them to ignore it, though, so the system cares very much.

OK. What is this fact supposed to teach us?

Re: OpenAI’s o1 correctly diagnosed 67% of ER patients vs. 50-55% by triage doctors

#448

Post by jmalicki »

n8henrie wrote: post #400
jmalicki wrote: post #347

Earlier quoted context omitted.

Because you can't order many of your own labs, and then insurance won't pay for them.

> you can't order many of your own labs Really? Which ones? > insurance won't pay for them Non sequitur, replacing doctors with AI will not help you pay for the preposterous US healthcare system. Vote!

> > you can't order many of your own labs > Really? Which ones?

There are extremely short lists of labs you can order yourselves. Virtually all of them are not on those short lists?

Re: OpenAI’s o1 correctly diagnosed 67% of ER patients vs. 50-55% by triage doctors

#449

Post by SauntSolaire »

ianbutler wrote: post #308
SauntSolaire wrote: post #275

Earlier quoted context omitted.

It's easy to destroy but hard to create. If your goal is to further destroy then I suppose that's achievable, but I have a hard time picturing what positive change is going to come from it.

No offense, but this comes off as passive indifference and while I've heard people say things like this all my life it has broadly resulted in watching 30 years of societal decay. I can't help but think this is wrong. We should have stacked the courts ourselves, brandished executive orders etc, had some spine. Edit: I think I need to make clear my thinking that the right has selectively destroyed institutions and lev…

Let's say, hypothetically, you had two political parties — a "destroy the current institutions" party, and the "preserve the current institutions" party.

The latter might notice the former having an easier time, but "hey, it works for them" is the wrong takeaway. Commit to the hard work of building resilient institutions; don't join in the destruction because it's easier.

There's also an element of "Never (...), they will drag you down to their level and beat you with experience."

Re: OpenAI’s o1 correctly diagnosed 67% of ER patients vs. 50-55% by triage doctors

#450

Post by mrguyorama »

ianbutler wrote: post #308
SauntSolaire wrote: post #275

Earlier quoted context omitted.

It's easy to destroy but hard to create. If your goal is to further destroy then I suppose that's achievable, but I have a hard time picturing what positive change is going to come from it.

No offense, but this comes off as passive indifference and while I've heard people say things like this all my life it has broadly resulted in watching 30 years of societal decay. I can't help but think this is wrong. We should have stacked the courts ourselves, brandished executive orders etc, had some spine. Edit: I think I need to make clear my thinking that the right has selectively destroyed institutions and lev…

"Stacking courts" would require a Senate that actually votes those judges in. "Brandishing Executive orders" requires a congress that won't be able to countermand you and a Supreme Court that won't "nuh uh" you.

You are yet another person upset that Democrats cannot overcome the purposeful design of our government that you need a lot of power to build, and little power to destroy.

People who want to fix things need dramatically more power than people who want to stymie and break things. Democrats only rarely get that power, and usually only by one or two votes from people who strictly do not care about fixing things. You want this country to fix things? You need to vote significantly more for a party who will push to fix things.

The minority party in congress has no power by design.

Post reply on HN