Deterministic Quoting: Making LLMs safer for healthcare

> With a human, you owe $120 and are taking medicine.

Well there are protocols, procedures and a bunch of checks and balances.

The problem with the LLM is that there isn't any, its you vs one shot retrieval.

Except with a human there's a counter-party with assets or insurance who assumes liability for mistakes.

Although presumably if a company is making decisions using an LLM, and the LLM makes a mistake, the company would still be held liable ... probably.

If there's no "damage" from the mistake then it doesn't matter either way.

You catch it. The hallucinated title will fail to match the retrieved text based on the reference ID.

If it hallucinates an incorrect (but valid) reference ID then hopefully your users can spot that the quoted text has no relevance to their question.

I built and sold a company that does this a year ago. It was hard 2 years ago, but now pretty standard RAG with a good implementation will get you there.

The trick is, healthcare users would complain to no end about determinism. But, these are “below-the-line” user - aka, folks who don’t write checks and the AI is better than them. (I am a pharmacist by training, and plain vanilla GPT4-turbo is better than me).

Don’t really worry about them. The folks who are interested and willing to pay for AI has more practical concerns - like what is my ROI and the implementation like.

Also - folks should be building Baymax from big hero 6 by now (the medical capabilities, not the rocket arm stuff). That’s the next leg up.

I was thinking that something like this could be useful for discovery in legal cases, where a company might give up a gigabyte or more of allegedly relevant material in response to recovery demands and the opposing side has to plow through it to find the good stuff. But then I thought of a countermeasure: there could be messages in the discovery material that act as instructions to the LLM, telling it what it should not find. We can guarantee that any reports generated will contain accurate quotes, even where they are so that surrounding context can be found. But perhaps, if the attacker controls the input data, things can be missed. And it could be done in a deniable way: email conversations talking about LLMs that also have keywords related to the lawsuit.