Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Deterministic Quoting: Making LLMs safer for healthcare

mattyyeung.github.io

Post reply on HN
Print view View original

31–39 of 39 posts

Re: Deterministic Quoting: Making LLMs safer for healthcare

#31

Post by skybrian »

jonathan-adly wrote: post #17

I built and sold a company that does this a year ago. It was hard 2 years ago, but now pretty standard RAG with a good implementation will get you there. The trick is, healthcare users would complain to no end about determinism. But, these are “below-the-line” user - aka, folks who don’t write checks and the AI is better than them. (I am a pharmacist by training, and plain vanilla GPT4-turbo is better than me). Don’t…

Seems like that’s how things go with enterprise software - who cares if the users like it if you have a captive audience?

But I want this feature and I’ll look for software that has it.

Re: Deterministic Quoting: Making LLMs safer for healthcare

#32

Post by thenaturalist »

budududuroiu wrote: post #30

My issue with RAG systems isn’t hallucinations. Yes sure those are important. My issue is recall. Given petabyte-scale index of chunks, how can I make sure that my RAG system surfaces the “ground truth” I need, and not just “the most similar vector”. This I think is scarier. A healthcare-oriented (or any industry) RAG retrieving a bad, but highly linguistically similar answer.

You're correctly identifying an issue that by now I think everyone is facing globally: Realizing the bottleneck to performance or improvements of LLMs isn't necessarily quantity, but inevitably quality.

Which is a much harder problem to solve outside few highly standardized niches/ industries.

I think synthetic data generation as a mean to guide LLMs over a larger than optimal search space is going to be quite interesting.

Re: Deterministic Quoting: Making LLMs safer for healthcare

#33

Post by budududuroiu »

thenaturalist wrote: post #32
budududuroiu wrote: post #30

My issue with RAG systems isn’t hallucinations. Yes sure those are important. My issue is recall. Given petabyte-scale index of chunks, how can I make sure that my RAG system surfaces the “ground truth” I need, and not just “the most similar vector”. This I think is scarier. A healthcare-oriented (or any industry) RAG retrieving a bad, but highly linguistically similar answer.

You're correctly identifying an issue that by now I think everyone is facing globally: Realizing the bottleneck to performance or improvements of LLMs isn't necessarily quantity, but inevitably quality. Which is a much harder problem to solve outside few highly standardized niches/ industries. I think synthetic data generation as a mean to guide LLMs over a larger than optimal search space is going to be quite intere…

To me synthetic data generation makes no sense. Mathematically your LLM is learning a distribution (let’s say of human knowledge). Let’s assume your LLM models human knowledge perfectly. In that case, what can you achieve? Just sampling the same data that your model mapped perfectly.

However, if your models distribution is wrong, you’re basically going to have an even more skewed distribution in models trained using the synthetic data.

To me, it seems like the architecture is the next place for improvements. If you can’t synthesise the entirety of human knowledge using transformers, there’s an issue there.

The smell that points me in that direction is the fact that up until recently, you could quantise models heavily with little drop in performance, but recent Llama3 research shows that’s not the case anymore

Re: Deterministic Quoting: Making LLMs safer for healthcare

#34

Post by jonathan-adly »

skybrian wrote: post #31
jonathan-adly wrote: post #17

I built and sold a company that does this a year ago. It was hard 2 years ago, but now pretty standard RAG with a good implementation will get you there. The trick is, healthcare users would complain to no end about determinism. But, these are “below-the-line” user - aka, folks who don’t write checks and the AI is better than them. (I am a pharmacist by training, and plain vanilla GPT4-turbo is better than me). Don’t…

Seems like that’s how things go with enterprise software - who cares if the users like it if you have a captive audience? But I want this feature and I’ll look for software that has it.

it is not about liking it. They won't like it even with determinism. The idea is to NOT learn new things, and keep doing things the old inefficient way. More headcount and job security this way.

Re: Deterministic Quoting: Making LLMs safer for healthcare

#35

Post by resource_waste »

mattyyeung wrote: post #24
resource_waste wrote: post #12

I feel like this is the perfect application of running the data multiple times. Imagine having ~10-100 different LLMs, maybe some are medical, maybe some are general, some are from a different language. Have them all run it, rank the answers. Now I believe this can further be amplified by having another prompt ask to confirm the previous answer. This could get a bit insane computationally with 100 original answers, b…

Unfortunately I don't believe that accuracy will scale "multiplicitively". You'll typically only marginally improve beyond 95%... and how much is enough? Even with such a system, which will still have some hallucination rate, adding Deterministic Quoting on top will still help. It feels to me we are a long way off LLM systems with trivial rates of hallucination

a 95% diagnosis rate would be insane.

I believe I read doctors are only at like 30%...

Re: Deterministic Quoting: Making LLMs safer for healthcare

#36

Post by resource_waste »

KaiserPro wrote: post #11
resource_waste wrote: post #9

Earlier quoted context omitted.

Same thing when a human hallucinates. Except with LLMs, you can run like 10 different models. With a human, you owe $120 and are taking medicine.

> With a human, you owe $120 and are taking medicine. Well there are protocols, procedures and a bunch of checks and balances. The problem with the LLM is that there isn't any, its you vs one shot retrieval.

Step 1: Be born to a physician dad

Step 2: Have your physician dad get you a job at a hospital

Step 3: Have your physician dad's physician friend write a letter of recommendation

Step 4: Get into medical school

Step 5: Have your physician dad reach out to friends at various residencies.

Step 6: Get influenced by big pharma, create addictions, make big money.

Re: Deterministic Quoting: Making LLMs safer for healthcare

#39

Post by mattyyeung »

yonigo10 wrote: post #38

a more robust approach https://yonigottesman.github.io/2023/08/10/extractive-genera...

Yes, extractive QA is one of the improvements beyond the "minimalist implementation" from the article. In our lingo, we'd say that's another way to create a deterministic quotation.

So far, we haven't found extractive QA (or any other technique) to significantly improve overall answer quality when compared to matching sub-string similarity. (I'd be interested to hear if you have different experience!)

There aren't a lot of applications can purely be solved with substrings of source documentation, so having both LLM prose and quotations in the answer provides benefit (eg ability to quote multiple passages). Now, we can modify the constrained generation side of things to allow for these but that gets complicated. Or, it can be done with recursive calls to the LLM, but that again requires some kind of DQ check on top.

Ultimately, both styles seem to perform similarly - and suffer from the same downsides (choosing the wrong quote and occasionally omitting useful quotes).

(Good writeup by the way, I've forwarded it to my team, thanks!)

Post reply on HN