Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A neurology ICU nurse on AI in hospitals

codastory.com

Post reply on HN
Print view View original

51–60 of 145 posts

Re: A neurology ICU nurse on AI in hospitals

#51

Post by stego-tech »

itishappy wrote: post #26
anthonyskipper wrote: post #15

Earlier quoted context omitted.

Some of us look forward to that future where you mostly just interact with AI. The one depressing us is not turning running over our goverment to AI. The sooner we can do that the better, you can't trust humans.

Can we trust AI?

Define “trust”, because that singular word carries immeasurable weight.

Can we trust AI to make consistent predictions from its training data? Yeah, fairly reliably. Can we trust that data to be impartial? What about the people training the model, can we trust their impartiality? What about the investors bankrolling it, can we trust them?

The more you examine the picture in detail, the less I think we’re able to state it’s trustworthy.

Re: A neurology ICU nurse on AI in hospitals

#52

Post by lekanwang »

As an investor in healthcare AI companies, I actually completely agree that there's a lot of bad implementations of AI in healthcare settings, and what practitioners call "alarm fatigue" as well as the feeling of loss of agency is a huge thing. I see a lot of healthcare orgs right now roll out some "AI" "solution" in isolation that raises one metric of interest, but fails to measure a bunch of other systemic measures.

Two thoughts: 1: I think the industry could take cues from aerospace and the human factors research that's drastically improved safety there -- autopilot and autoland systems in commercial airliners are treated as one part of a holistic system with the pilot and first officer and flight attendants in keeping the plane running smoothly. Too few healthcare AI systems are evaluated holistically.

2: Similarly, if you're going to roll out a system, either there's staff buy-in, or the equilibrium level of some kind of quality/outcomes/compliance measure should increase that justifies staff angst and loss of agency. Not all AI systems are bad. One "AI" company we invested in, Navina, is actually loved by physicians using them, but the team also spent a LOT of time doing UX research and feedback with actual users and the support team is always super responsive.

Re: A neurology ICU nurse on AI in hospitals

#53

Post by heironimus »

This is the same technology story told thousands of times a day with nearly every technology. Medical seems to be especially bad at this.

Take a very promising technology that could be very useful. Jump on it early without even trying to get buy in and without fully understanding the people that will use it. Then push a poor version of it.

Now the nurses hate the tech, not the poor implementation of it. The techies then bypass the nurses because they are difficult, even though they could be their best resource for improvement.

Re: A neurology ICU nurse on AI in hospitals

#54

Post by cowmix »

This article feels “ripped from today’s headlines” for me, as my mother-in-law was recently in the ICU after a fall that caused head trauma. The level of AI-driven automated decision-making is unsettling, especially as it seems to allow large organizations to deflect accountability—“See? The AI made us do it!” I’m not entirely sure what guided her care—or lack thereof—but, as someone who frequently works in healthcare IT, I see these issues raised all the time.

On the other hand, having access to my own “AI” was incredibly helpful during her incident. While in the ICU, speaking with her doctors, I used ChatGPT and Claude to become a better advocate for her by asking more informed questions. I could even take pictures of the monitors tracking her vitals, and ChatGPT helped me interpret the readings, which was surprisingly useful.

In this “AI-first” world we’re heading into, individuals need their own tools to navigate the asymmetric power dynamic with large organizations. I wonder how long it will be until these public AI models get “tweaked” to limit their effectiveness in helping us question “the man.”

Re: A neurology ICU nurse on AI in hospitals

#55

Post by bluefirebrand »

PhasmaFelis wrote: post #33
ToucanLoucan wrote: post #3

Earlier quoted context omitted.

> AI converging with medical technology is a really dangerous space, for obvious reasons. An article like this does make me worry it's being rushed through, but not because of the author's objections, instead because of their ignorance of what is and what isn't AI I mean, worth pointing out that OpenAI has been shoving LLMs into people's faces for going on about a year and half now at global scale and calling it AI,…

That's been the technical definition in the AI research community for at least 50 years. "AI = machines that think like people" is the sci-fi definition.

> AI = machines that think like people" is the sci-fi definition

It's also the layman's definition

Which does matter because laymen are the ones who are treating this current batch of AI as the silver bullet for all problems

Re: A neurology ICU nurse on AI in hospitals

#56

Post by rtkwe »

anthonyskipper wrote: post #15
coliveira wrote: post #10

Everyone should be terrified. The "promise" of AI is the following: remove any kind of remaining communication between humans, because that is "inefficient", and replace it with an AI that will mediate all human interactions (in business and even in other areas). In a few years, AIs trained by big corps will run the show and humans will be required to interface with them to do anything of value. Similar to what they…

Some of us look forward to that future where you mostly just interact with AI. The one depressing us is not turning running over our goverment to AI. The sooner we can do that the better, you can't trust humans.

That's still trusting humans, either the ones who created the AI and gave it it's goals/parameters or the humans that actually implement it's edicts. Can't get away from people, it's a lesson all the DAO hype squad learned quickly, fundamentally you still need people to implement the decisions.

Re: A neurology ICU nurse on AI in hospitals

#57

Post by luxcem »

andy_ppp wrote: post #30
maxehmookau wrote: post #21

Earlier quoted context omitted.

> Some of us look forward to that future where you mostly just interact with AI. What is it about that that appeals to you? I'm genuinely curious. A world without human interaction feels like a world I don't want to exist in.

If you are autistic (for example) I'm guessing human interaction can be extremely difficult and very stressful and triggering. Machines are much more amenable and don't have loads of arbitrary unwritten rules the way humans do. Maybe the idea of being entrapped by bureaucracy introduced by machines will be better than the the bureaucracy introduced by humans?

> Machines are much more amenable and don't have loads of arbitrary unwritten rules

I'm sure system prompts of the most famous LLM are just that

Re: A neurology ICU nurse on AI in hospitals

#58

Post by A_D_E_P_T »

croes wrote: post #46
A_D_E_P_T wrote: post #29

Earlier quoted context omitted.

Counterpoint: AI is actually better at communication than most humans. In fact, even an ancient (in relative terms) article found that AI bots have better bedside manner than human doctors: https://www.theguardian.com/technology/2023/apr/28/ai-has-be... Today, I expect it's not even very close. I also believe that AI diagnostics are on average more accurate than the mean human doctor's diagnostic efforts -- and can b…

AI is better in simulating communication but worse in understanding. >you can do the same thing with GPT-4o and get a reasonable response in no time at all -- at and no cost. Reasonable doesn't mean correct. Who is liable if it's the wrong answer?

"AI" is basically a vast, curated, compressed database with a powerful index. If the database reflects the current state of the art, it'll have better understanding than the majority of human practitioners.

You may say it will "simulate understanding" -- but in this case the simulation would be indistinguishable from the real thing, thus it would be the real thing. (Really "indiscernible" in the philosophical sense of the word.)

> Reasonable doesn't mean correct. Who is liable if it's the wrong answer?

I think that you can get better accuracy than with the average human doctor. Beyond that, my own opinion is that liability should be quisque pro se.

Re: A neurology ICU nurse on AI in hospitals

#59

Post by SpaceLawnmower »

fhfjfk wrote: post #14
bryanlarsen wrote: post #9

Earlier quoted context omitted.

> If it sucks and hallucinates information it's not going to go live anywhere. It hallucinates and it's live in many places. My doctor uses it. AFAICT the hallucination rate is fairly low. It's transcription and summarization which has a lower hallucination rate than when it's asked for an answer. It's massively better than the alternative. Taking the time to write down proper notes is the difference between 4 patien…

What about increasing the supply of doctors rather than decreasing the time they spend with patients? I question the underlying premise that efficiency needs to increase.

Note taking is not about the time spent with patients. It's about keeping a good record for next time and insurance and is a major reason for physician burnout. Some doctors will finish up charting when after hours.

Yes physicians could still see fewer patients but filling out their mandatory notes is annoying regardless of it's a manageable amount of patients or a work extra hours amount.

Re: A neurology ICU nurse on AI in hospitals

#60

Post by boohoo123 »

100% agree AI will ruin healthcare. I'm an IT director at a rural mental health clinic and I see the push for AI across my state and it's scary what they want. All i can do is push back. Healthcare is a case by case personal connection, something AI can't do. It only reduces humans down to numbers and operates on that. There is no difference between healthcare AI to a web scraper on webmd or mayo clinic.
Post reply on HN