Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A neurology ICU nurse on AI in hospitals

codastory.com

Post reply on HN
Print view View original

131–140 of 145 posts

Re: A neurology ICU nurse on AI in hospitals

#131

Post by frm88 »

stego-tech wrote: post #51
itishappy wrote: post #26

Earlier quoted context omitted.

Can we trust AI?

Define “trust”, because that singular word carries immeasurable weight. Can we trust AI to make consistent predictions from its training data? Yeah, fairly reliably. Can we trust that data to be impartial? What about the people training the model, can we trust their impartiality? What about the investors bankrolling it, can we trust them ? The more you examine the picture in detail, the less I think we’re able to sta…

>What about the investors bankrolling it, can we trust them?

This is the crucial question. We live in capitalism and maximising profits is the most dominant axiom.

Without any legislation in place anywhere in the world and AI-supported diagnosis and therapy very much already in place what would prevent the companies bankrolling the software/hardware from bricking the tool on various grounds, like payments not made, political reasons: Cory Doctorow wrote a blog-post in 2022 where John Deere bricked Ukrainian tractors [0]; or a manufacturer of respirators refused to repair the devices at the height of COVID, so that only Hackers enabled the devices to run and save human lifes? [1]

[0] https://doctorow.medium.com/about-those-kill-switched-ukrain...

[1] https://www.hackster.io/news/polish-hacker-shares-software-s...

Re: A neurology ICU nurse on AI in hospitals

#132

Post by bangaroo »

A_D_E_P_T wrote: post #104
bangaroo wrote: post #101

Earlier quoted context omitted.

Read the conclusions section from the paper you linked: > GPT-4o’s performance in USMLE disciplines, clinical clerkships, and clinical skills indicates substantial improvements over its predecessors, suggesting significant potential for the use of this technology as an educational aid for medical students. These findings underscore the need for careful consideration when integrating LLMs into medical education, empha…

I get that you really disdain LLMs. But consider that a totally off-the-shelf, stock model is acing the medical licensing exam. It doesn't only perform better than human counterparts at the very peak of their ability (young, high-energy, immediately following extensive schooling and dedicated multidisciplinary study) it leaves them in the dust. If you think that the test is simple or even text-only, here are some sam…

it has nothing to do with disdain of LLMs, i'm an extensive user of warp (a very good LLM-based tool) and at my job we use them in depth in the software i build for summarization and other tasks that LLMs are generally considered good at. i spend a lot of time working with LLMs and find that, in some cases, they can be extremely useful, particularly when it comes to completing simple tasks in natural language.

i am also aware of their limitations and have a reasonable and realistic view of what they can currently do and where they are headed. i have seen many failure modes, i am familiar with patterns in their output, and i understand the boundaries of their comprehension, capabilities and understanding.

not buying into the current silicon valley money pit du jour and misunderstanding studies to validate that view does not equate to just being disdainful. i'm being realistic, because i understand what they do and how they work.

i'm not going to circle with you - you don't seem all that interested in engaging with the meat of anything i say to you, and instead just want to continue to try and rationalize your misunderstanding of the single study you found in support of your position, which is your right.

i feel ethically obligated to say, once again, that an LLM isn't a doctor and you should under no circumstances go to one for medical advice. you could really cause yourself some problems.

if you do so, that's on you. best of luck. incidentally i suspect someone has an awesome picture of a monkey at a steep discount you might be interested in.

Re: A neurology ICU nurse on AI in hospitals

#133

Post by hartspear »

lekanwang wrote: post #52

As an investor in healthcare AI companies, I actually completely agree that there's a lot of bad implementations of AI in healthcare settings, and what practitioners call "alarm fatigue" as well as the feeling of loss of agency is a huge thing. I see a lot of healthcare orgs right now roll out some "AI" "solution" in isolation that raises one metric of interest, but fails to measure a bunch of other systemic measures…

Agreed, what was obviously missing from this person's experience was an opportunity to guide deployment in ways that work well for their understanding of the workflow. I think companies often fail to explore what actually happens on the ground of their organizations and make assumptions that are not accurate. AI could be a huge help to nurses but it takes conversations and direct insight during development to get it right.

Re: A neurology ICU nurse on AI in hospitals

#134

Post by chubot »

danudey wrote: post #94
chubot wrote: post #65

Earlier quoted context omitted.

It's true, but corporate policies and insurance are already like "slow AI" They remove most of what's real in interactions I remember going for a routine checkup at Kaiser, and the doctor was literally checking boxes on her computer terminal, rather than looking, talking, listening. I dropped them after that -- it was pointless for me to go It seems like there are tons of procedures that already have to be followed,…

> I remember going for a routine checkup at Kaiser, and the doctor was literally checking boxes on her computer terminal, rather than looking, talking, listening. This is also a doctor issue, to be clear. My primary care physician has a program he uses on his laptop; I'm not sure what program it is, but he's been using it since I started going to him around 2009 so it's definitely not something new. He goes through a…

Yes definitely, the terminal doesn't necessarily imply doing a bad job... and I'm sure it can improve it. But I guess I'm saying I could see this unfortunate person being worn down by bureaucracy, not really engaged. Also I saw similar issues with the nurses.

Re: A neurology ICU nurse on AI in hospitals

#135

Post by boohoo123 »

moralestapia wrote: post #117
boohoo123 wrote: post #60

100% agree AI will ruin healthcare. I'm an IT director at a rural mental health clinic and I see the push for AI across my state and it's scary what they want. All i can do is push back. Healthcare is a case by case personal connection, something AI can't do. It only reduces humans down to numbers and operates on that. There is no difference between healthcare AI to a web scraper on webmd or mayo clinic.

I'm not vouching for AI, I actually thing it will only make things worse. But, >Healthcare is a case by case personal connection [...] I haven't felt this with doctors in like 20 years.

I get that but thats from the corporatization of clinics. They push for numbers not client well being. They want doctors to meet with x amount of patients a day and doctors dont control their schedule. So typically the execs meet together and say if we schedule every 20 minutes instead of 30 minutes we can generate x amount of more dollars. Which is why doctors just get you in and out as fast as possible because its their head on the chopping block if they dont meet the measures. Best thing you can do is find smaller clinics, which are becoming increasingly rare because the cost it takes to run one.

Re: A neurology ICU nurse on AI in hospitals

#136

Post by bitwize »

ToucanLoucan wrote: post #71
bitwize wrote: post #17

Earlier quoted context omitted.

The simple state machines and decision trees that make enemies move and attack in video games, we also call AI. AI is a loose term and always has been. It's like the term "robot": we call any machine a robot which either a) resembles a human or part of one (e.g., a robotic arm), or b) can perform some significant human labor that involves decision making (e.g., a robot car that drives itself). Similarly, AI is anythi…

> The simple state machines and decision trees that make enemies move and attack in video games, we also call AI. Yes but no game companies have ever marketed their game asserting that the things your shooting are, in fact, self-aware conscious intelligences.

Tell me you weren't around for the PlayStation 2 marketing blitz without telling me you weren't around for the PlayStation 2 marketing blitz.

They actually named the thing's CPU the Emotion Engine.

Re: A neurology ICU nurse on AI in hospitals

#137

Post by glonq »

boohoo123 wrote: post #60

100% agree AI will ruin healthcare. I'm an IT director at a rural mental health clinic and I see the push for AI across my state and it's scary what they want. All i can do is push back. Healthcare is a case by case personal connection, something AI can't do. It only reduces humans down to numbers and operates on that. There is no difference between healthcare AI to a web scraper on webmd or mayo clinic.

Like any tool, the right amount of AI in the right places could be used to improve healthcare, and wrong/wrong could be used to ruin it.

If your healthcare system is in the hands of politicians and/or capitalists, then expect the worst.

Re: A neurology ICU nurse on AI in hospitals

#138

Post by A_D_E_P_T »

bangaroo wrote: post #132
A_D_E_P_T wrote: post #104

Earlier quoted context omitted.

I get that you really disdain LLMs. But consider that a totally off-the-shelf, stock model is acing the medical licensing exam. It doesn't only perform better than human counterparts at the very peak of their ability (young, high-energy, immediately following extensive schooling and dedicated multidisciplinary study) it leaves them in the dust. If you think that the test is simple or even text-only, here are some sam…

it has nothing to do with disdain of LLMs, i'm an extensive user of warp (a very good LLM-based tool) and at my job we use them in depth in the software i build for summarization and other tasks that LLMs are generally considered good at. i spend a lot of time working with LLMs and find that, in some cases, they can be extremely useful, particularly when it comes to completing simple tasks in natural language. i am a…

It's far more than a single study, just one example of a very powerful development. The same phenomenon is also occurring in state bar exams, etc. And that one study is hardly misunderstood -- as you can verify for yourself.

> i understand the boundaries of their comprehension, capabilities and understanding.

It seems to me that you are quite far behind the current state of the art, and you apparently underestimate even stock GPT-o1, which is pretty old news.

I'm willing to place a friendly wager with you: Let's find a doctor who does online consultations and give him three questions selected at random from that sample test. These are diagnostic-type questions that well reflect what a country doctor would encounter in daily practice. We can leave the questions open-ended or give him the multiple-choice options. Then we put GPT-o1 to the same questions. I'd be very happy to bet that the LLM outperforms the doctor. I'd even place a secondary bet that the LLM answers all questions correctly and that the doctor answers less than two questions correctly.

Re: A neurology ICU nurse on AI in hospitals

#139

Post by edanm »

theptip wrote: post #64

> As a nurse, you end up relying on intuition a lot. It’s in the way a patient says something, or just a feeling you get from how they look There is a longstanding tension between those who believe human intuition is trustworthy, and the “checklist manifesto” folks. Personally I want room for both, there are plenty of cases where for example the nurse/doctor’s intuition fails and they forget to ask about travel or ou…

> There is a longstanding tension between those who believe human intuition is trustworthy, and the “checklist manifesto” folks.

I really think this is a false dichotomy. Both are needed.

I even vaguely remember from the Checklist Manifesto book, that one of the checklist items was specifically designed to make sure people actually talk to each other and share plans/thoughts/concerns before things like surgeries, which seems like a great way to make sure that intuition gets heard.

Re: A neurology ICU nurse on AI in hospitals

#140

Post by dmz73 »

A_D_E_P_T wrote: post #123
A_D_E_P_T wrote: post #104

Earlier quoted context omitted.

I get that you really disdain LLMs. But consider that a totally off-the-shelf, stock model is acing the medical licensing exam. It doesn't only perform better than human counterparts at the very peak of their ability (young, high-energy, immediately following extensive schooling and dedicated multidisciplinary study) it leaves them in the dust. If you think that the test is simple or even text-only, here are some sam…

As an aside, I quickly tested GPT-o1 by giving it question 95 on the sample test. I'm no doctor, but I've got extensive training in chemistry and biochem, and I'm not ashamed to admit that the question totally stumped me. GPT-o1 gave the correct answer the first time around, and a very detailed explanation as to why all other potential answers must be false. A really remarkable performance, I think. Now imagine it's…

LLM can only provide statistically most likely diagnosis based on the training data. If used by the experienced doctor LLM can be a valuable tool that saves time and maybe even increases accuracy in majority of cases. The problem is that LLMs will be used to replace experienced doctors and will be used as 100% accurate tools. This will result in experienced doctors becoming first rare and then non-existent and outcomes for patients will become increasingly unfavorable due to LLMs always producing a confident result even when it would be obviously wrong to an experienced doctor or even just a student.
Post reply on HN