Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Demand for human radiologists is at an all-time high

worksinprogress.news

Post reply on HN
Print view View original

121–130 of 486 posts

Re: Demand for human radiologists is at an all-time high

#121

Post by philwelch »

mandevil wrote: post #94
kstrauser wrote: post #18

Earlier quoted context omitted.

What’s your take on pharmacists? To my naive eyes, that seems like a certainty for replacement. What extra value does human judgement bring to their work?

My wife is a clinical pharmacist at a hospital. I am a SWE working on AI/ML related stuff. We've talked about this a lot. She thinks that the current generation of software is not a replacement for what she does now, and finds the alerts they provide mostly annoying. The last time this came up, she gave me two examples: A) The night before, a woman in her 40's came in to the ER suffering a major psychological breakdo…

> And it needs to be someone different from the person filling out the Rx, for Swiss cheese model of safety reasons.

This is something I question. If you go to a specialist, and the specialist judges that you need surgery, he can just schedule and perform the surgery himself. There’s no other medical professional whose sole job is to second-guess his clinical judgment. If you want that, you can always get a second opinion. I have a hard time buying the argument that prescription drugs always need that second level of gatekeeping when surgery doesn’t.

Re: Demand for human radiologists is at an all-time high

#122

Post by m0llusk »

christianqchung wrote: post #70
kklisura wrote: post #54

When Tesla demoed (via video) self-driving in 2016 with a claim "The person in the driver’s seat is only there for legal reasons. He is not doing anything. The car is driving itself" and then when they unveiled Semi in 2017 - I tweeted out and honestly thought that trucking industry is changed forever and it doesn't make sense to be starting in trucking industry. It's almost end of 2025 and either nothing out of it o…

Look at Waymo, not Robotaxi. Waymo is essentially the self driving vision I had as a kid, and ridership is growing exponentially as they expand. It's also very safe if you believe their statistics[0]. I think there's a saying about overestimating stuff in the short term and underestimating stuff in the long term that seems to apply here, though the radiologist narrative was definitely wrong. [0] https://waymo.com/saf…

Waymo is very impressive, but also demonstrates limitations of these systems. Waymo vehicles are still getting caught performing unsafe driving maneuvers, they get stuck alleys in numbers, and responders have trouble getting them to acknowledge restricted areas. I am very supportive of this technology, but also highly skeptical as long as these vehicles are directly causing problems for me personally. Driving is more than a technical challenge, it involves social communication skills that automated vehicles do not yet have.

Re: Demand for human radiologists is at an all-time high

#123

Post by nitwit005 »

roncesvalles wrote: post #48
djoldman wrote: post #10

> Three things explain this. First,... Second, attempts to give models more tasks have run into legal hurdles: regulators and medical insurers so far are reluctant to approve or cover fully autonomous radiology models. Third, even when they do diagnose accurately, models replace only a small share of a radiologist’s job. Human radiologists spend a minority of their time on diagnostics and the majority on other activi…

I only came to this thread to say that this is completely untrue: >Human radiologists spend a minority of their time on diagnostics and the majority on other activities, like talking to patients and fellow clinicians. The vast majority of radiologists do nothing other than: come in (or increasingly, stay at home), sit down at a computer, consume a series of medical images while dictating their findings, and then go h…

What the article suggests is backed up by research, at least in hospital settings: https://www.jacr.org/article/S1546-1440(13)00220-2/abstract

Re: Demand for human radiologists is at an all-time high

#124

Post by jmhmd »

ineedasername wrote: post #79

>"they struggle to replicate this performance in hospital conditions" Are there systematic reasons why radiologists in hospitals are inaccurately assessing the AI's output? If the AI models are better than humans in testing novel data then, well, the thing that has changed in a hospital situation compared to the AI-Human testing environment is not the AI, it is the human, under less controlled constraints, additional…

Poorer performance in real hospital settings has more to do with the introduction of new/unexpected/poor quality data (i.e. real world data) that the model was not trained in or optimized for. They still do very well generally, but often do not hit equivalent performance to what is submitted to the FDA, or in marketing materials. This does not mean they aren’t useful.

Clinical AI also has to balance accuracy with workflow efficiency. It may be technically most accurate for a model to report every potential abnormality with associated level of certainty, but this may inundate the radiologist with spurious findings that must be reviewed and rejected, slowing her down without adding clinical value. More data is not always better.

In order for the model to have high enough certainty to get the right balance of sensitivity and specificity to be useful, many many examples are needed for training, and with some rarer entities, that is difficult. It also inherently reduces the value of the model it is only expected to identify its target disease 3 times/year.

That’s not to say advances in AI won’t overcome these problems, just that they haven’t, yet.

Re: Demand for human radiologists is at an all-time high

#126

Post by 1vuio0pswjnm7 »

kklisura wrote: post #54

When Tesla demoed (via video) self-driving in 2016 with a claim "The person in the driver’s seat is only there for legal reasons. He is not doing anything. The car is driving itself" and then when they unveiled Semi in 2017 - I tweeted out and honestly thought that trucking industry is changed forever and it doesn't make sense to be starting in trucking industry. It's almost end of 2025 and either nothing out of it o…

"I think we all have become hyper-optimistic on technology. We want this tech to work and we want it to change the world in some fundamental way, but either things are moving very slowly or not at all."

Who is "we"? The people who hype "AI"?

Re: Demand for human radiologists is at an all-time high

#127

Post by ineedasername »

nomel wrote: post #83
ineedasername wrote: post #79

>"they struggle to replicate this performance in hospital conditions" Are there systematic reasons why radiologists in hospitals are inaccurately assessing the AI's output? If the AI models are better than humans in testing novel data then, well, the thing that has changed in a hospital situation compared to the AI-Human testing environment is not the AI, it is the human, under less controlled constraints, additional…

For anomaly systems like this, is it effective to invert the problem by not include the ailment/problem in the training data, then looking for a "confused" signal rather than a "x% probability of ailment" type signal?

On that, I'm not sure. My area of ML & data science practice is, thankfully, not so high-stakes. There's a method of anomaly detection called one-class SVM (Support Vector Machine) that is pretty much this- train on normal, flag on "wtf is this you never training me on this 01010##" Not actual ISO standard ML model output or medical jargon. But I'm not sure if that's what's most effective here. My gut instinct in first approaching the task would be to throw a bunch of models at it, mixed-methods, with one-class SVM as a fall back. But I'm also way out of my depth on medical diagnostics ML so that's just a generalist's guess.

Re: Demand for human radiologists is at an all-time high

#128

Post by jmhmd »

CamperBob2 wrote: post #111
pjdesno wrote: post #103

The best story I heard about machine learning and radiology was when folks were racing to try to detect COVID in lung X-rays. As I recall, one group had fairly good success, but eventually someone figured out that their data set had images from a low-COVID hospital and a high-COVID hospital, and the lettering on the images used different fonts. The ML model was detecting the font, not the COVID. [a bit of googling la…

Anecdotes like this are informative as far as they go, but they don't say anything at all about the technique itself. Like your story about the fonts used for labeling, essentially all of the drawbacks cited by the article come down to inadequate or inappropriate training methods and data. Fix that, which will not be hard from a purely-technical standpoint, and you will indeed be able to replace radiologists. Sorry,…

The issue is that in medicine, much like automobiles, unexpected failure modes may be catastrophic to individual people. “Fixing” failure modes like the above comment is not difficult from a technical standpoint, that’s true, but you can only fix it once you’ve identified it, and at that point you may have a dead person/people. That’s why AI in medicine and self driving cars are so unlike AI for programming or writing and move comparatively at a snails pace.

Re: Demand for human radiologists is at an all-time high

#129

Post by philwelch »

mandevil wrote: post #115
philwelch wrote: post #89

Earlier quoted context omitted.

For trucking I think self driving can be, in the short term, an opportunity for owner-operators. An owner-operator of a conventional truck can only drive one truck at a time, but you could have multiple self driving trucks in a convoy led by a truck manned by the owner-operator. And there might be an even greater opportunity for this in Europe thanks to the low capacity of European freight rail compared to North Amer…

I used to think this sort of thing too. Then a few years ago I worked with a SWE who had experience in the trucking industry. His take was that most trucking companies are too small scale to benefit from this. The median trucking operation is basically run by the owner's wife in a notebook or spreadsheet- and so their ability to get the benefits of leader/follower mileage like that just doesn't exist. He thought that…

Yeah, I guess the addressable market of “truck owners who can afford to buy another truck but not hire another driver” might be smaller than I thought.

Re: Demand for human radiologists is at an all-time high

#130

Post by mandevil »

philwelch wrote: post #121
mandevil wrote: post #94

Earlier quoted context omitted.

My wife is a clinical pharmacist at a hospital. I am a SWE working on AI/ML related stuff. We've talked about this a lot. She thinks that the current generation of software is not a replacement for what she does now, and finds the alerts they provide mostly annoying. The last time this came up, she gave me two examples: A) The night before, a woman in her 40's came in to the ER suffering a major psychological breakdo…

> And it needs to be someone different from the person filling out the Rx, for Swiss cheese model of safety reasons. This is something I question. If you go to a specialist, and the specialist judges that you need surgery, he can just schedule and perform the surgery himself. There’s no other medical professional whose sole job is to second-guess his clinical judgment. If you want that, you can always get a second op…

So, the main reason for the historical separation (in the European tradition) between doctor and pharmacist was profit motive- you didn't want the person prescribing to have a financial stake in their treatment, else they will prescribe very expensive medicine for all cases. And surgeons in particular do have a profit motive- they are paid per service- and it is well known within the broader medical community that surgeons will almost always choose to cut. And we largely gate-keep this with the primary care physician providing a recommendation to the specialist. The PCP says "this may be something worth treating with surgery" when they recommend you go see a specialist rather than prescribing something themselves, and then the surgeon confirms (almost always).

That pharmacists also provide a safety check is a more modern benefit, due to their extensive training and ability to see all of the drugs that you are on (while a specialist only knows what they have prescribed). And surgeons also have a team to double-check them while they are operating, to confirm that they are doing the surgery on the correct side of the body, etc. Because these safety checks are incredibly important, and we don't want to lose them.

Post reply on HN