Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: How are you preventing LLM hallucinations in production systems?

news.ycombinator.com

Post reply on HN
Print view View original

11–16 of 16 posts

Re: Ask HN: How are you preventing LLM hallucinations in production systems?

#11

Post by kundan_s__r »

stephenr wrote: post #10
kundan_s__r wrote: post #9

Earlier quoted context omitted.

That’s fair — if you’re already skeptical and paying attention, the failures are obvious and often funny. The risk tends to show up more with non-experts or downstream systems that assume the output is trustworthy because it looks structured and confident. Autocomplete failing loudly is annoying; autocomplete failing quietly inside automation is where things get interesting.

> The risk tends to show up more with non-experts This hits a key point that isn't emphasised enough. A few interactions with technology and people have shaped my view: I fiddled with Apple's Image Playground thing sometime last year, and it was quite rewarding to see a result from a simple description. It wasn't exactly what I'd asked for, but it was close, kind of. As someone who has basically zero artistic ability…

The real risk with LLMs isn’t when they fail loudly — it’s when they fail quietly and confidently, especially for non-experts or downstream systems that assume structured output equals correctness.

When you don’t already understand the domain, AI feels infallible. That’s exactly when unvalidated outputs become dangerous inside automation, decision pipelines, and production workflows.

This is why governance can’t be an afterthought. AI systems need deterministic validation against intent and execution boundaries before outputs are trusted or acted on — not just better prompts or post-hoc monitoring.

That gap between “sounds right” and “is allowed to run” is where tools like Verdic Guard are meant to sit.

Re: Ask HN: How are you preventing LLM hallucinations in production systems?

#12

Post by raw_anon_1111 »

Let me give you a little anecdote. I use ChatGPT to learn Spanish. The prompt I use is below.

It gets things wrong about half the time and I have to tell it that it’s wrong. If I can’t trust an LLM to follow simple instructions, why would I trust it “agentically” with business critical decision making?

I work in cloud consulting specializing in app dev and every project I’ve done in the last year and a half has a bedrock based LLM somewhere in the process - ie the running system. But I know what to trust it for and what not to trust it for and I guide my clients appropriately.

The prompt I use for studying Spanish that ChatGPT gets wrong:

—- I am learning Spanish at a A2 level. When I ask you to do a lightning round, I will give you a list of sentences first. You will give me each English sentence one by one and I will translate it to Spanish. If I get it wrong, save it for the next round.

When I ask you to create sentences from a verb, create 1 sentences each for 1-3 single and 1 and 3 plural for present and simple past and 3 for progressive. Each sentence must be at least five words.

These are some words and phrases I need to review: only use these words in sentences for 1-3 present single and only when they make sense, If a target word does not fit naturally, skip it and prioritize a natural sentence. don’t force yourself to use these words. When I list of verb, it means I need to practice it, present and simple past

Never use:

Re: Ask HN: How are you preventing LLM hallucinations in production systems?

#13

Post by Agent_Builder »

kundan_s__r wrote: post #3
al_borland wrote: post #2

I’ve just been ignoring my boss every time he says something about how we should leverage AI. What we’re building doesn’t need it and can’t tolerate hallucinations. They just want to be able to brag up the chain that AI is being used, which is the wrong reason to use it. If I was forced to use it, I’d probably be writing pretty extensive guardrails (outside of the AI) to make sure it isn’t going off the rails and the…

That’s a very sane stance. Treating LLM output as untrusted input is probably the correct default when correctness matters. The worst failures I’ve seen happen when teams half-trust the model — enough to automate, but still needing heavy guardrails. Putting the checks outside the model keeps the system understandable and deterministic. Ignoring AI unless it can be safely boxed isn’t anti-AI — it’s good engineering.

[dead]

Re: Ask HN: How are you preventing LLM hallucinations in production systems?

#14

Post by kundan_s__r »

raw_anon_1111 wrote: post #12

Let me give you a little anecdote. I use ChatGPT to learn Spanish. The prompt I use is below. It gets things wrong about half the time and I have to tell it that it’s wrong. If I can’t trust an LLM to follow simple instructions, why would I trust it “agentically” with business critical decision making? I work in cloud consulting specializing in app dev and every project I’ve done in the last year and a half has a bed…

That’s a very real example of the core problem: LLMs don’t reliably honor constraints, even when they’re explicit and simple. Instruction drift shows up fast in learning tasks — and quietly in production systems.

That’s why trusting them “agentically” is risky. The safer model is to assume outputs are unreliable and validate after generation.

I’m working on this exact gap with Verdic Guard (verdic.dev) — treating LLM output as untrusted input and enforcing scope and correctness outside the model. Less about smarter prompts, more about predictable behavior.

Your Spanish example is basically the small-scale version of the same failure mode.

Re: Ask HN: How are you preventing LLM hallucinations in production systems?

#15

Post by kundan_s__r »

Agent_Builder wrote: post #13
kundan_s__r wrote: post #3

Earlier quoted context omitted.

That’s a very sane stance. Treating LLM output as untrusted input is probably the correct default when correctness matters. The worst failures I’ve seen happen when teams half-trust the model — enough to automate, but still needing heavy guardrails. Putting the checks outside the model keeps the system understandable and deterministic. Ignoring AI unless it can be safely boxed isn’t anti-AI — it’s good engineering.

[dead]

That framing resonates a lot. In production, creativity is often just unbounded variance.

Once each step is intentionally boring and constrained, failures become predictable and debuggable — which is what engineering actually optimizes for. That tradeoff is almost always worth it.

I’m building Verdic Guard (verdic.dev) around the same idea: treat LLMs as creative generators, but enforce scope and correctness outside the model so systems stay calm under load.

Re: Ask HN: How are you preventing LLM hallucinations in production systems?

#16

Post by crosslayer »

+1 to treating LLM output as untrusted input.

The failure mode I keep seeing isn’t hallucination per se… it’s blurred responsibility between intent and execution. Once a model can both decide and act, you’ve already lost determinism.

The stable pattern seems to be- LLMs generate proposals, deterministic systems enforce invariants. Creativity upstream, boring execution downstream.

What matters in production isn’t making models smarter, it’s making failure modes predictable and auditable. If you can’t explain why a state transition was allowed, the architecture is already too permissive.

Post reply on HN