Live data from Hacker News

OpenAI’s o1 correctly diagnosed 67% of ER patients vs. 50-55% by triage doctors

theguardian.com

471–480 of 500 posts

Re: OpenAI’s o1 correctly diagnosed 67% of ER patients vs. 50-55% by triage doctors

#471
post #211

Earlier quoted context omitted.

This is because "dismantle and replace" (or perhaps in other words, "defunding") is not a serious, viable solution to many of the societal issues we face. Things were ruined slowly. They unfortunately will need to be fixed very slowly too.

I don't think that's going to work. We need broad political change and then that has to work rapidly to legislate this. I don't think slow and steady has done anything but lead to the decay our institutions over the last 70 years.

I think that both this and GP are misguided. The pace of societal change in a given direction is neither inherently proportional to the pace of change in a different direction (GP) nor is the pace part of the direction (you).

You have to engage with the specific historical events/factors that led to the direction and the pace in order to change either. Broad statements like "society is big so change has to be slow" are just as unwarranted as "slow change results in decline".

There's a correct answer to "how quickly can change in a new direction be achieved". It will probably only become known after the fact. It will certainly not be model-able as a function with variables for "progressive or not" and "speed of change".

Re: OpenAI’s o1 correctly diagnosed 67% of ER patients vs. 50-55% by triage doctors

#472

Earlier quoted context omitted.

It's easy to destroy but hard to create. If your goal is to further destroy then I suppose that's achievable, but I have a hard time picturing what positive change is going to come from it.

No offense, but this comes off as passive indifference and while I've heard people say things like this all my life it has broadly resulted in watching 30 years of societal decay. I can't help but think this is wrong. We should have stacked the courts ourselves, brandished executive orders etc, had some spine. Edit: I think I need to make clear my thinking that the right has selectively destroyed institutions and lev…

Strongly agree. I think some (not all) of the Trumpian playbook can be wielded very effectively for non-conservative parties, for a few reasons:

- Some executive orders are always flipped as soon as the opposition takes office, but some unilateral changes are much harder for a cyclical/pendulum-swing opposition season to reverse than they are to emplace. We don't know which are which yet. The return-to-office mandate for Federal workers is probably one that'll have a lasting effect--even if un-done in the future, the average prospective Federal worker will consider the job as something that has a significant likelihood of requiring in-person work if the political winds change and that EO is restored.

- Some things really do get permanently addressed within an electoral season, if you have the guts to shotgun through enacting a solution to them. The withdrawal of most U.S. troops from Afghanistan under Biden is a good example of this. So is the "Fork"/RIF/firing wave of Federal employees under Trump. I'm not saying those are both good things, but they aren't "reversible" in the sense that, say, the Global Gag Rule was endlessly reversible.

- Success follows success, as well. Part of the reason that momentum holds such a sacred place in electoral planning is the same reason that Trump's "flood the zone" strategy was effective (again--not good, but undeniably effective): capitalizing on/marketing early unilateral wins of any size results in the public and Congress being more likely to support larger, more durable changes. This is complicated by many factors (media landscape, districting, money), but is broadly true.

Re: OpenAI’s o1 correctly diagnosed 67% of ER patients vs. 50-55% by triage doctors

#473
post #124

Earlier quoted context omitted.

Doctors talk to patients? I know. I know. Part of it is that talking to patients on average is useless but still this can’t be really used for an argument against AI. Still doctors can have a more broad picture of the situation since they can look at the patient as a whole; something the LLM can’t really synthesize in its context.

There’s really nothing preventing an LLM from having the context a doctor does. The two avenues of context gathering by the doctor are: - looking at their medical history/charts - asking follow up questions An LLM based system is trivially capable of doing both of those.

> looking at their medical history/charts

I think you'd be incredibly surprised how often charts are super, super incomplete or wrong. Like "pt has no pancreas and presented with pain and weeping from a 6yo pancreatectomy scar" but the chart doesn't mention the surgery or the entire missing organ wrong. Like "pt is a twin whose sibling died traumatically of cancer in front of them a year ago and presents with probable hypochondria about cancer" but the chart doesn't mention any family history wrong. Like "lifelong history of severe cognitive impairment substantiated by a psych eval; attended annual physical before being sent to imaging for head trauma because of observed impairment" but the chart doesn't mention cognition (someone was too polite to note it) nor the psych eval (records sharing wasn't allowed) wrong.

Those are a very few examples off the top of my head. I worked in EMR. I don't know shit about medicine, but man, do I know a lot about the complaints physicians and their staff send when they think it's the records system's fault that the chart was wrong or missing info.

In a big chunk of cases, the MD/NP/whatever's in-person role is determining what's not on the chart so that they can then ask appropriate follow-up questions. Given the massive range of possible dx for a given issue, and how much of getting the right dx doesn't have to do with probabilities/numbers of similar patients with the same symptom:dx data that'd be in the training set, I have major doubt that an LLM can appropriately intuit or appropriately question in order to diagnose.

Re: OpenAI’s o1 correctly diagnosed 67% of ER patients vs. 50-55% by triage doctors

#474

Earlier quoted context omitted.

I would argue that the ED is the least similar to code. You have the most unknowns, unreliable data and history, non deterministic options and time constraints. An ER staff is frequently making inferences based on a variety of things like weather, what the pt is wearing, what smells are present, and a whole lot of other intangibles. Frequently the patients are just outright lying to the doctor. An AI will not pick up…

> An AI will not pick up on any of that. It will if it trains on data like that. It's all about the training data.

To give this more credit than it perhaps deserves: training aside, getting the situational data into the context is a more significant problem here.

Pt's chart is complex/wrong? Gotta ingest that into context.

Chart contains images/scanned and not OCR'd text? Gotta do an image recognition pass.

Diagnosis needs to know what the pt's wearing (i.e. radiation badge)? Gotta do an image recognition pass.

Diagnosis needs to know what the weather's like? Internet API access of some kind. Hope the WAN/API are all working! If they're not, do you fail open or closed?

Patient might be lying? Gotta do video/audio analysis to assess that likelihood--oh, and train a model that fully solves one of the holy grails of computer vision/audio analysis reliably and with a super low false-positive rate before you do. And if it guesses wrong, enjoy the incredibly easy-to-prosecute lawsuit.

Patient might be lying, but the biggest clue is e.g. smell of alcohol on their breath? Now you need some sort of olfactory sensor kit and training for it--a lot more than just "low quality body cam and a mic".

Patient's ODing on a street drug that became abundant in the last few months? Gotta somehow learn about recent local medical/police history that post-dates the training set, or else you might be pouring gas on a fire if you give them Narcan. And that's assuming you know enough to search for information about that drug, and that they didn't lie to you about what they took. Addicts never do that.

Failures in each of those systems bring down the chance of an effective diagnosis, so they need a fairly obsessive amount of model introspection/thinking/double-checking, and humans on standby as a fallback if the AI's less than confident (assuming that LLMs can be given a sense of a confidence level in the future, versus the current state of the art of "text-predict a guess about what your confidence level might be").

Put that all together, and even with the AI compute speed available years from now and a perfectly trained futuristic model that's preternaturally good at this stuff, I'm not sure that that the reliability and, more importantly, the turnaround time of that diagnostic pass is going to be any good compared to a human ER doc.

Re: OpenAI’s o1 correctly diagnosed 67% of ER patients vs. 50-55% by triage doctors

#475

Earlier quoted context omitted.

It is statements like this that convince me we haven't learned anything and are doomed to ever wider pendulum swings.

I think the time for the normal decorum and extended hand have passed.

I wonder if your political opponents see things similarly. Types like these, a theory of mind is especially useful.

Re: OpenAI’s o1 correctly diagnosed 67% of ER patients vs. 50-55% by triage doctors

#476

Earlier quoted context omitted.

I think the time for the normal decorum and extended hand have passed.

I wonder if your political opponents see things similarly. Types like these, a theory of mind is especially useful.

Of course they do. Most of their platform is built on [appearing] to repudiate coastal elitism and left wing dogma in higher education + globalism with a healthy dose of fuck you because you're you.

And I graciously waited and allowed them to do things that will take decades if not more to repair before deciding they were irredeemable. I had hoped a middle ground and bipartisan ship would be reached, but it's clear to me it won't be. We do not inhabit the same universe at this point, the disdain is mutual.

You’re acting like I’ve always thought about them like this or like I haven't spent years observing and thinking about this to come to my conclusion. You'd do well to listen to your own words about theory of mind. I was raised conservative I voted for Romney. I'm a fan of many of the political platforms they run on now (minus originalism, removing bodily rights, religion), but in practice they do not walk their own talk. The wars, the spending, abandoning neo-liberalism except in word the blatant corruption and disdain for the positions they hold and how they appear on the world stage.

No, I’ve watched their actions for 15 years and moved ever closer to the position that I have nothing in common with them even being ideologically close to a version of their party from 20-30 years ago and they do so blatantly want to destroy the middle class, health, and wealth for anyone outside a small oligarchic class.

I'm pissed because they wear a lot of my ideology as cheap dress to fuck someone.

Re: OpenAI’s o1 correctly diagnosed 67% of ER patients vs. 50-55% by triage doctors

#477

Earlier quoted context omitted.

I don't think that's going to work. We need broad political change and then that has to work rapidly to legislate this. I don't think slow and steady has done anything but lead to the decay our institutions over the last 70 years.

I think that both this and GP are misguided. The pace of societal change in a given direction is neither inherently proportional to the pace of change in a different direction (GP) nor is the pace part of the direction (you). You have to engage with the specific historical events/factors that led to the direction and the pace in order to change either. Broad statements like "society is big so change has to be slow" a…

My argument is more along the lines of "slow change has resulted in decline observably for the time period I have observed it and we should try catalyzing something else"

I grant that whether that winds up being fast or slow even if the attempt is intended to be fast is out of my or anyone's hands for the most part as the system dampens that barring total collapse and chaos :P

Re: OpenAI’s o1 correctly diagnosed 67% of ER patients vs. 50-55% by triage doctors

#478

Earlier quoted context omitted.

You are confusing the job with a subset of tasks. Some tasks can be automated, some won't. That doesn't mean LLMs, which cannot tell how many r's are in strawberry, will replace anyone.

I'm not. I understand the difference and also that through improvements to the core models as well as harnesses, LLMs are able to handle an increasing share of tasks. I also understand that these things will continue to improve until AI can automate entire jobs. You, on the other hand, are confusing LLMs from the past with current SOTA LLMs, which can tell how many rs are in strawberry.

It's tech bros like you that are to blame for the shortage of radiologists supporting statements since 2016 which state that radiologists will disappear. Your great SOTA LLMs will tell you to walk 5 mins to the car wash instead of taking the car.

Re: OpenAI’s o1 correctly diagnosed 67% of ER patients vs. 50-55% by triage doctors

#479

Earlier quoted context omitted.

You are confusing the job with a subset of tasks. Some tasks can be automated, some won't. That doesn't mean LLMs, which cannot tell how many r's are in strawberry, will replace anyone.

> That doesn't mean LLMs, which cannot tell how many r's are in strawberry, will replace anyone. But most of us live in America in 2026. There are a lot of interests that don't give a shit about you who would love if you to got your medical care from a machine that "cannot tell how many r's are in strawberry". And there a lot of useful idiots with no real medical issues who will loudly claim the machine is great.

I cannot agree more. Useful idiots and people working in private equity which have a direct financial interest to hype this tech.

Re: OpenAI’s o1 correctly diagnosed 67% of ER patients vs. 50-55% by triage doctors

#480
post #413

Earlier quoted context omitted.

Does she think this really does the complexity of each case justice though? I doubt you can compress an anamnesis into a two-liner without losing essential data.

> Does she think this really does the complexity of each case justice though? Do you believe that -prior to the 2020-ish mass evacuation of doctors from the profession- the typical specialist would misrepresent the facts of a case when asking for a cross-check? Related: Have you ever worked as "the guys who actually work on the thing"-level tech support for a nontrivial Enterprise Software Product (or System)? If you…

An enterprise product is not comparable with the human body at all. A single cell contains hundreds of times more information/entropy in its state than an operating system.
Post reply on HN