Hallucination Risk Calculator • Hacker News Board

Re: Hallucination Risk Calculator

#41

Post by K0balt » Thu, Sep 11, 2025, 1:22 PM UTC

toss1 wrote: post #38

weinzierl wrote: post #27

Earlier quoted context omitted.

It sometimes happens with simple things. I once pasted the announcement for an event in Claude to check for spelling and grammar. It had a small suggestion for the last sentence and repeated the whole corrected version for me to copy and paste. Only last sentence slightly modified - or so I thought because it had moved the date of the event in the first sentence by one day. Luckily I caught it before posting, but it…

Yup, I always take editing suggestions and implement them manually, then re-feed the edited version back in for new suggestions if needed. Never let it edit your stuff directly —— the risk of stealth random errors sneaking in is too great. Just because every competent human we know would edit ONLY the specified parts, or move only the specified columns with a cut/paste operation (or similar deterministically reliable…

Tool use seems like a much better solution in theory. I wonder how it works out IRL?

Re: Hallucination Risk Calculator

#42

Post by ep103 » Thu, Sep 11, 2025, 3:20 PM UTC

epiccoleman wrote: post #39

ep103 wrote: post #19

Earlier quoted context omitted.

Yesterday I used ChatGPT to transform a csv file. Move around a couple of columns, add a few new ones. Very large file. It got them all right. Except when I really looked through the data, for 3 of the excel cells, it clearly just made up new numbers. I found the first one by accident, the remaining two took longer than it would have taken to modify the file from scratch myself. Watching my coworkers blindly trust ou…

I don't mean to be rude, but this sounds like user error. I don't understand why anyone would use an LLM for this - or at least, why you would let the LLM perform the transformation. If I was trying to do something like this I would ask the LLM to write a Python script, validate the output by running it against the first handful of rows (like, `head -n 10 thing.csv | python transform-csv.py`). There are times when st…

Because it markets and presents itself as deterministic and honest. That's the whole issue. AI is unethically marketed and presented to the public.

Re: Hallucination Risk Calculator

#43

Post by epiccoleman » Fri, Sep 12, 2025, 1:15 AM UTC

ep103 wrote: post #42

epiccoleman wrote: post #39

Earlier quoted context omitted.

I don't mean to be rude, but this sounds like user error. I don't understand why anyone would use an LLM for this - or at least, why you would let the LLM perform the transformation. If I was trying to do something like this I would ask the LLM to write a Python script, validate the output by running it against the first handful of rows (like, `head -n 10 thing.csv | python transform-csv.py`). There are times when st…

Because it markets and presents itself as deterministic and honest. That's the whole issue. AI is unethically marketed and presented to the public.

iPod marketing presented then as a device that made you cool. I just used mine to listen to music though

Re: Hallucination Risk Calculator

#44

Post by gaussdiditfirst » Tue, Sep 16, 2025, 5:21 PM UTC

MontyCarloHall wrote: post #15

spindump8930 wrote: post #12

This topic is interesting, but the repo and paper have a lot of inconsistencies that make me think this work is hiding behind lots of dense notation and language. For one, the repo states: > This implementation follows the framework from the paper “Compression Failure in LLMs: Bayesian in Expectation, Not in Realization” (NeurIPS 2024 preprint) and related EDFL/ISR/B2T methodology. There doesn't seem to be a paper by…

It's telling that neither the repo nor the linked paper have a single empirical demonstration of the ability to predict hallucination. Let's see a few prompts and responses! Instead, all I see is a lot of handwavy philosophical pseudo-math, like using Kolmogorov complexity and Solomonoff induction, two poster children of abstract concepts that are inherently not computable, as explicit algorithmic objectives.

Ya I saw no comparison with other methods in the paper, which is odd for a ML paper.