11–20 of 46 posts
“That’s real, and it’s interesting.” Stop with the slop.
People are not going to stop spamming you with slop text unless the narrative is "Using AI for writing is a mark of poor literacy/low intelligence".
There are a whole host of new EMRs popping up that aim to help clinicians make judgement calls about how to answer certain questions and even when particular procedures are relevant. The last one we demoed, each piece of information it retrieved from the LLM has a confidence score attached to it. Our nurses have to fill out 200+ question forms when taking on a new patient that all HAVE to be completed in a single-go meaning that you can't split it up into multiple forms and it's gotta be one cohesive unit.
Imagine being a nurse with little technical skill and almost no idea how these tools work trying to make sense of what the difference between a 90% and 70% is across 200 different questions. "We're 60% sure the patient is allergic to nuts" means jack shit to them. Granted, sometimes the scores are complimented with actual references in the underlying documentation (history and physical, referring info) but sometimes it's not.
So there are objective ways to control hallucinations as well as figuring out how “correct” the answer is to some extent.
The easiest argument in favour of numeric probability is that it's simple to evaluate. Someone claiming to be 90 % certain better be right about nine out of ten times -- no fewer, but also not too often! This is useful when you're learning to judge your own confidence, which most people are bad at. (Almost everyone can train themselves to be better at i but some people seem to be good at it with no training.)
The typical argument against categories is that people can mean very different things with the same word: https://i.ibb.co/kQT4Ymz/q.png
It seems like TFA gets around these problems by creating a translation table between a fixed set of categories and the probabilities implied by the model for those categories. That's an interesting approach!
This resonates with what I'm seeing in healthcare right now and the bad taste it's leaving. There are a whole host of new EMRs popping up that aim to help clinicians make judgement calls about how to answer certain questions and even when particular procedures are relevant. The last one we demoed, each piece of information it retrieved from the LLM has a confidence score attached to it. Our nurses have to fill out 20…
“That’s real, and it’s interesting.” Stop with the slop.
Bender: I can guarantee anything you want.
Who can read that text color and background combo? I had to turn on reader mode in Firefox.
I use the "Dark Background and Light Text" add-on. Sometimes the add-on makes a bad website worse (like IMDB), but it can be switched off with a click.
This resonates with what I'm seeing in healthcare right now and the bad taste it's leaving. There are a whole host of new EMRs popping up that aim to help clinicians make judgement calls about how to answer certain questions and even when particular procedures are relevant. The last one we demoed, each piece of information it retrieved from the LLM has a confidence score attached to it. Our nurses have to fill out 20…
Healthcare still uses fax. No surprise they arent using Frontier AI to answer these questions. (And if you still don't trust AI, have the report on the left side of the screen and the source pages on the right side of the screen)
