Deterministic Quoting: Making LLMs safer for healthcare
mattyyeung.github.io
Deterministic Quoting: Making LLMs safer for healthcare
1–10 of 39 posts
Re: Deterministic Quoting: Making LLMs safer for healthcare
#2We’ve developed LLM W^X now - time to develop LLM ROP!
Re: Deterministic Quoting: Making LLMs safer for healthcare
#3We’ve developed LLM W^X now - time to develop LLM ROP!
Interesting analogies for LLMs! (https://en.wikipedia.org/wiki/W%5EX & https://en.wikipedia.org/wiki/Return-oriented_programming)
Re: Deterministic Quoting: Making LLMs safer for healthcare
#4I'm not sure determinism alone is sufficient for proper attribution.
This presumes "chunks" are the source. But it's not easy to identify the propositions that form the source of some knowledge. In the best case, you are looking for an association and find it in a sentence you've semantically parsed, but that's rarely the case, particularly for medical histories.
That said, deterministic accuracy might not matter if you can provide enough context, particularly for further exploration. But that's not really "chunks".
So it's unclear to me that tracing probability clouds back to chunks of text will work better than semantic search.
Re: Deterministic Quoting: Making LLMs safer for healthcare
#5Did I miss something or did the article never describe how the technique works? (Despite the “How It Works” section
Re: Deterministic Quoting: Making LLMs safer for healthcare
#6Did I miss something or did the article never describe how the technique works? (Despite the “How It Works” section
It's explained at considerable length in the section _A “Minimalist Implementation” of DQ: a modified RAG Pipeline_.
Re: Deterministic Quoting: Making LLMs safer for healthcare
#7It's a search engine, basically?
Re: Deterministic Quoting: Making LLMs safer for healthcare
#8What happens if it hallucinates the ?
Re: Deterministic Quoting: Making LLMs safer for healthcare
#9What happens if it hallucinates the ?
Same thing when a human hallucinates.
Except with LLMs, you can run like 10 different models. With a human, you owe $120 and are taking medicine.
Re: Deterministic Quoting: Making LLMs safer for healthcare
#10It's a search engine, basically?
A good, automatically run, privacy preserving search engine that uses electronic medical records might be a valuable resource for busy doctors.