Do we need deterministic AI?

#1

Post by createdbymason » Fri, Jul 11, 2025, 5:30 AM UTC

deterministic LLMs - what are they good for? do we need them? someone explain ...

Re: Do we need deterministic AI?

#2

Post by handfuloflight » Fri, Jul 11, 2025, 5:39 AM UTC

Prompt: "Output a name of one of the states of the United States."

It will not tell me the names of the planets. Or the names of prescription drugs.

Why is this level of determinism not enough?

Re: Do we need deterministic AI?

#3

Post by theGeatZhopa » Fri, Jul 11, 2025, 5:54 AM UTC

handfuloflight wrote: post #2

Prompt: "Output a name of one of the states of the United States." It will not tell me the names of the planets. Or the names of prescription drugs. Why is this level of determinism not enough?

at some point one might need determined responses. RAG & Law, or in certain situations, where RAG is done on f.e. legal or some constraints are in hold. LLMs shouldn't respond non-deterministic then.

But this is solveable without deterministic AI. Nonetheless, to have some deterministic AI would solve some pain in some areas you & me not work in, but others.. ;)

Re: Do we need deterministic AI?

#4

Post by handfuloflight » Fri, Jul 11, 2025, 6:08 AM UTC

theGeatZhopa wrote: post #3

handfuloflight wrote: post #2

Prompt: "Output a name of one of the states of the United States." It will not tell me the names of the planets. Or the names of prescription drugs. Why is this level of determinism not enough?

at some point one might need determined responses. RAG & Law, or in certain situations, where RAG is done on f.e. legal or some constraints are in hold. LLMs shouldn't respond non-deterministic then. But this is solveable without deterministic AI. Nonetheless, to have some deterministic AI would solve some pain in some areas you & me not work in, but others.. ;)

I also work in those situations and I don't see how prompting with a grounded context isn't giving the determinism needed?

Re: Do we need deterministic AI?

#5

Post by rvz » Fri, Jul 11, 2025, 6:17 AM UTC

Of course we do.

Reliability is more valuable than anything.

Would you want the AI system to mistake the sun for a yellow warning sign whilst you are driving on the highway?

Re: Do we need deterministic AI?

#6

Post by theGeatZhopa » Fri, Jul 11, 2025, 2:02 PM UTC

handfuloflight wrote: post #4

theGeatZhopa wrote: post #3

Earlier quoted context omitted.

at some point one might need determined responses. RAG & Law, or in certain situations, where RAG is done on f.e. legal or some constraints are in hold. LLMs shouldn't respond non-deterministic then. But this is solveable without deterministic AI. Nonetheless, to have some deterministic AI would solve some pain in some areas you & me not work in, but others.. ;)

I also work in those situations and I don't see how prompting with a grounded context isn't giving the determinism needed?

may be your needing of determinism isn't everyones needing of determinism and you never worked on those situations where prompting with a grounded context didn't give the determinism needed??

may be you should talk to this guys:

[1] SteadyText: Deterministic LLMs: Same input → same output, every time (julep.ai) -> https://news.ycombinator.com/item?id=44458132 (7d ago, 0 comments)

[2] Enhancing LLM performance with reasoning using deterministic feedback loops (usekbai.com) -> https://news.ycombinator.com/item?id=44482032 (4d ago, 0 comments)

[3] The AI Agent schism: deterministic vs. non deterministic (kunle.app) -> https://news.ycombinator.com/item?id=44379994 (7h ago, 0 comments)

I think, [1], [2] and [3] would be happy to discuss and gladly learn from your experience. That's all about discussion - learn from others & teach some others. I'll follow them and look forward to learn from the discussions and your arguments, too.