Live data from Hacker News

Kaiser nurses say AI, surveillance are making their jobs and patient care worse

localnewsmatters.org

371–380 of 401 posts

Re: Kaiser nurses say AI, surveillance are making their jobs and patient care worse

#371

Earlier quoted context omitted.

What makes them stupid?

It's stupid because there's no formal definition of what counts as AI, so we repeat the same conversation every time. Hence the joke ML is just AI that works. There's a book from 1995, called Artificial Intelligence: A Modern Approach, by by Stuart Russell and Peter Norvig, which gives the definition "AI is the study of agents that receive percepts from the environment and perform actions." but doesn't define intelli…

I personally would find it sufficient just to prove both that AI can exist without ML and that ML can exist without AI. The former is easy to prove, since AI has essentially existed since even before computing, while the latter is harder to prove, since ML as a field was created for AI.

> If we go with that, though, I'd say AI is anything that uses a model to make decisions instead of hand written "if" statements.

Hand-written if-statements are a model though...

I would say AI is anything with the intention of performing as a human does. In my opinion, LLMs are AI because they're designed to either perform a human role, or be addressable like a human to a user. (In general I do not agree with implying intelligence by humanity or that it is exclusive to humanity, but it is genuinely the easiest way for me to explain this here.)

Even LLMs that are used as part of harnesses or backend services would count for this, because you could technically replace the model with an equivalent human and they would be able to read and write in its place.

And pre-LLM AI would also count for this, because the whole selling point of AI is that it's like a human in some way or could perform as a human does in some way. Even if it's purely for entertainment value or whatever, if the entertainment value includes that it seems like a human.

But uses of ML that have nothing to do with this would not count, like for example camera image enhancement. You could say a human could sit there and work through some algorithm manually (like from a book, etc) but that's not what image enhancement algorithms are doing. Now if you let an LLM have access to human-UX-ful photo editing tools and let it iterate or etc. then that would be AI but that's another thing.

It's difficult to define precisely exactly what this definition is though. Different people have different ideas of what "performing as a human does" means. If the ML algorithm is executing machine code instructions to perform a task then could a human execute the same machine code instructions by hand to prove that it's AI? No, because the human has to be in place of it, not emulating it. The human would have to take its inputs and give its outputs, without depending on its definition. So, then, it depends on where you define the boundaries of the system. LLMs typically output in tokens, so would a human have to think in tokens? No, because the use of tokens is as a text encoding, and humans are perfectly free to use tools in their work, etc. A human being able to read and write text in English would still prove that a language model that reads and writes the same but in tokens is AI as long as they would be more or less interchangeable.

Of course, today's LLMs are not really true replacements for humans but I'm not talking about performance here, just where they can be positioned. AI to me is more of an intention than a technology -- any technology can be AI depending on how you use it. ML however is a technology, no matter the intention.

Re: Kaiser nurses say AI, surveillance are making their jobs and patient care worse

#372

Earlier quoted context omitted.

Being close relative for several med and care workers we have discussed it a lot and consensus is that it really depends. For example relying on LLM summaries sounds great until it doesn't. It doesn't matter whether you misunderstand LLM summary or LLM "misunderstands" you – there are real risks involved, and you wouldn't want them to weigh on your conscience if they were to materialize. Relying on LLM to summarize t…

> It doesn't matter whether you misunderstand LLM summary or LLM "misunderstands" you – there are real risks involved, and you wouldn't want them to weigh on your conscience if they were to materialize. "fun" fact, a number (some? many? all?) of the LLM speech to text models will editorialize or hallucinate words for you, to make your speech fit the pattern it expects. I don't think it's appropriate to use a model th…

In a medical setting we first have to be clear about whether the LLM is acting as a transcriptionist or a scribe. Those are different roles. A transcriptionist merely writes down everything the clinician says verbatim, with perhaps a bit of formatting to fit a standard template. Speech recognition software has been used for transcription since before LLMs even existed. Sometimes a human transcriptionist will review and edit the software output. But ultimately the clinician is accountable for approving the document before it legally becomes part of the patient chart.

A medical scribe isn't just taking dictation, they're doing some level of interpretation and often entering structured, coded data directly into an EHR. This is a more complex role and requires some clinical skills. There are some new LLM products that automate this to an extent, but ultimately the licensed clinician is still legally accountable for what goes into the patient chart.

I can absolutely ethically argue for rationing care. All healthcare systems ration care although the means vary. Demand is effectively infinite, especially from older patients with complex or terminal conditions. Resources are finite.

Re: Kaiser nurses say AI, surveillance are making their jobs and patient care worse

#373
post #317

Earlier quoted context omitted.

There is still a categorical difference between how they are being used. Specifically analytic vs generative. Generative AI (LLMs and image generators) are the ones people have issues with - pretty much nobody cares about ML processing for analysis.

> pretty much nobody cares about ML processing for analysis. I work in a bank and a can tell you that the customers absolutely hate ML when it rejects their loan application. Over the pond in the US, I have an impression that the fico score is not exactly popular either, but I have no first hand experience.

In the US, FICO scores are mainly unpopular with "credit criminals" who have low scores. The score performs quite well in predicting how likely a borrower is to repay a loan. The problems that arise with credit scores in general are when they're used for purposes they were never designed to serve, like screening job applicants.

For home mortgage lending, FICO scores are now being partially replaced by VantageScore.

https://vantagescore.com/

Re: Kaiser nurses say AI, surveillance are making their jobs and patient care worse

#374
post #249

Earlier quoted context omitted.

I personally will not use a provider who uses llm tools. I know it makes me and my coworkers less careful and lazier. Qualities I dont want in a health care providor. Ive actually moved primary care physicians over this once already, found the oldest guy I could who barely knows how to use a laptop but spends a bunch of extra time with me.

Likewise. I basically - will not see a “provider”. A doctor (MD/DO) only please. - will not see a doctor who uses LLM tools (or therapists for that matter) - Make it abundantly clear that if you do then we’re not a match. - Will pay significantly more/go out of my way for for this.

Regardless of whether your preferences are rational or not, you're quite lucky to be able to pick and choose. There is a shortage of clinicians and it's only going to get worse so most patients will have to take whomever they can get, especially if they're not able to wait months for an appointment.

Re: Kaiser nurses say AI, surveillance are making their jobs and patient care worse

#375
post #161
post #3

If you think using a machine to evaluate how well a human is showing empathy is a good idea, you probably shouldn't have any position of power.

If a human’s performance for 40 plus hours a week can be reduced to a performance score , that too in a field like healthcare, that alone feels weird. Sure there needs to be some way to reward people/evaluate their work etc, but I dunno if management by metrics is the best way, especially when those numbers are calculated by an algorithm. Where has all the empathy gone? And common sense? :(

I agree with you, but in huge organizations like Kaiser Permanente there doesn't seem to be any good solution. For management at scale there are multiple levels of organizational hierarchy between the decision makers and the individual contributors (like nurses directly dealing with patients). So they need some kind of systematic way to compare employees who work under different managers. Just relying on managers to rate their own direct reports based on empathy and common sense doesn't work well at scale, either.

Re: Kaiser nurses say AI, surveillance are making their jobs and patient care worse

#376

Earlier quoted context omitted.

> The patient load hadn't grown. The number of nurses has gone up, not down. Healthcare utilization, number of unique patients, amount of care per patient has all gone up. American has a massive overutilization problem driving cost but is unwilling to be honest with itself about it.

AKA the "Be your own advocate and demand more testing!" crowd. Although Kaiser tends to repel such patients (they'll happy say "No" and tell you to find some other provider). The bulk of complaints I've heard about KP in California boil down to "They wouldn't give me a referral to a specialist".

Yes, Kaiser is a great model. We need to figure out a way to balance cost sharing with preventing over utilization, both on the supply side and managing demand and restricting interventions to those that are measurably changing outcomes (generally drugs, less so human-capital intensive care).

Re: Kaiser nurses say AI, surveillance are making their jobs and patient care worse

#377

Earlier quoted context omitted.

Now let's talk about how AI could replace Kaiser middle management and streamline bureaucracy to preserve more budget for doctors and nurses...

People enjoy saying "AI can replace middle management" as a comeuppance but it's not what people think. In many ways, AI is primarily serving as middle management, yes. But what happens is that instead of one manager engaging in petty surveillance, harassment and browbeating to increase work intensity, you have ten clones of the manager doing the same thing. That's not the "woo hoo, automated middle management!" you…

Understood and agreed. I admit I was feeling a bit pithy with that comment.

I shouldn't really have referenced "replace" at all, but rather "optimize". And not targeting middle management but the whole non-medical structure above. This is where I think most of the waste in our modern US medical system lives.

But I understand it is still mostly a half-baked concept. We'd need a revolution far outside of just some management system. I believe it's the whole enclosing system that is rotten and driving medical costs away from the actual encounter costs. Profiteering, risk management, regulatory capture, etc.

I woudl argue there was a point, long ago, where these things have taken on a life of their own and no longer serve the need of improving care outcomes and the actual medical cost-benefit tradeoff we should be optimizing in a just and caring society.

Re: Kaiser nurses say AI, surveillance are making their jobs and patient care worse

#378
post #331

Earlier quoted context omitted.

> Today its a mixed bag of corpo hate, anxiety about the future, inequality and traditional class warfare, combined with the typical technical ignorance. Are there no actual problems with what it promises/how we're expected to use it at work/how it is used against us by people who use it at work? Is it just anxieties and feelings, that aren't actually grounded in anything? Everything's getting better for everyone, no…

In this case it seems to me that LLMs in medicine pretty much are just better for everyone with no real downsides. Im not a doctor but have 2 in my family I speak with regularly and from how they talk about it care is pretty much just unequivocally better now because they dont have to be sitting at the computer taking notes all the time. It removed a huge part of the process that neither patients nor doctors enjoyed.

Are prices going down, as they need to do less work per patient?

Or are the providers just pocketing the difference?

Given that prices are definitely not going down, I'm not sure how this makes my life any better.

Re: Kaiser nurses say AI, surveillance are making their jobs and patient care worse

#379

Earlier quoted context omitted.

isn’t that how we got here though? Everyone in silicon valley was so busy making sure they were open minded, steelmanning, and not treating politics like a sport that no one in the silicon valley halls of power pointed out that the emperor has no clothes and now we have a (sorry woke police) shrieking retard president leading us through the singularity.

It definitely seems like having Trump in power suits the rich and powerful tech lobby just fine, that's for sure

Thats’s the trump pitch, that supporting him is defecting in a prisoners dilemma, but from where I stand it looks like he’a shrinking the pie by damaging America through ineptitude faster than he’s increasing the fraction of the pie available to all but his closest cronies.

Re: Kaiser nurses say AI, surveillance are making their jobs and patient care worse

#380

Earlier quoted context omitted.

Let me try this again. A two party system is a rigged game. Period. The Democratic party absolutely needs to be taken to task and none of their bullshit should be tolerated or defended, ever, except one small exception: election day, in the general election -- because we can't fix the government if it's no longer a democracy.

Thank you for clarifying your position. We are in violent agreement in all but this: I refuse to reward the Democratic party for putting a gun to my head. If they as a political organization are willing to literally risk our democracy when the alternative is assembling reforms that clearly address the imbalances in our society then we can all burn together because I refuse to continue to reward their brinksmanship.

Great! I engage to try and reach consensus, not dismiss. Sometimes it even works!

I want to point out that there were enough viable voters for Harris to win (but I am in the camp that believes there was actual election fraud done by Trump supporters).

The anger at blessing Harris without a proper primary, the genocide in Gaza, and the "everything's fine in the economy because the market is doing great!" -- none of that should be tolerated, and I recognize the anger leading to not voting for her because of that.

But by not voting in that unpalatable compromise we have an administration that is as evil, corrupt, and anti-democratic as could be. It's not clear if we can ever wrest control again. I'm frankly terrified.

We have to be strategic in our opposition, but that involves all of us reading from the same script -- a power that the Right excels at and the Left seems incapable of doing.

Case in point: "wokeness". It's been artfully weaponized by the Right and is the Achilles heel of the Left. It's the GOP's golden ticket to victory.

One of the superpowers of "wokeness" is that it's definition is never acknowledged on the Right other than "shrill libruls trying to boss everyone around and make everybody transgender and gay". That vagueness is intentional. To the Right it's an opaque blob of hate that requires no thought, no reason, other than "I hate that shit".

As "woke" was really about becoming aware of the institutionalized racism of the US I would call myself "fully woke" in that regard. But as it's more of a amorphous entity now I'd call it "human decency".

That said, I believe the Dems need to bail on "woke" and follow the model of the Tea Party, where a subgroup can espouse the more "extreme" stuff, and the main party can say "oh those silly people, that's not us, we're the serious party of populist economics and foreign policy"; then once that is framed in the public's eye, to go "oops, the extremists made me do this".

Politics means 2 distinctly different things: tending to the rules of governance, and tribal warfare. It's the latter that makes people hate talking about it.

This is all a game, a very important game, and a very strategic game. The rules used to be more clear but now its Calvinball.

Our only viable choice is to play the game the best we can, and recognize and act on what will help us "win".

Disclaimer: while I shit on the GOP, I recognize the value of "conservatism" and would love to see a GOP that existed to be the check and balance of any follies that the Dems enage in. If we're stuck with a two-party system, that would be our best bet for the country to survive, let alone succeed.

Post reply on HN