[dead]
Don't ask an LLM for a confidence score
41–46 of 46 posts
Re: Don't ask an LLM for a confidence score
#42This resonates with what I'm seeing in healthcare right now and the bad taste it's leaving. There are a whole host of new EMRs popping up that aim to help clinicians make judgement calls about how to answer certain questions and even when particular procedures are relevant. The last one we demoed, each piece of information it retrieved from the LLM has a confidence score attached to it. Our nurses have to fill out 20…
these can work as long as the evidence is attached (to your point). And in healthcare absolutely must (your EMR example is terrifying). The patient may be allergic to nuts (confidence .84) >> cite: 1) lab value X, PRO Y, ... >> otherwise I'm 62% confident I'm an elephant.
Re: Don't ask an LLM for a confidence score
#43Using a 0-100 scale is a pretty bad idea in general. Neither humans not models know what to do with it. Split your metric into several concrete variables with discrete values well understood by the user (human or model). If you still need something more granular after that, calculate the aggregate score.
Re: Don't ask an LLM for a confidence score
#44You don’t ask an LLM, but certain LLMs expose internal metrics you can tell how many token candidates where there what was the score which one and which one was selected. So there are objective ways to control hallucinations as well as figuring out how “correct” the answer is to some extent.
But those scores are only the likelihood of the given token being the next one in a plausible phrase, not that the underlying data is correct.
Yes that is correct but it’s still better than nothing, there are some assumptions to be made such as that the model was not trained on junk data.
Re: Don't ask an LLM for a confidence score
#45[dead]
Re: Don't ask an LLM for a confidence score
#46[deleted]