really interesting approach to calibration for hallucinations, im going to give this a go on some of my projects.
Hallucination Risk Calculator
11–20 of 44 posts
Re: Hallucination Risk Calculator
#12This topic is interesting, but the repo and paper have a lot of inconsistencies that make me think this work is hiding behind lots of dense notation and language. For one, the repo states:
> This implementation follows the framework from the paper “Compression Failure in LLMs: Bayesian in Expectation, Not in Realization” (NeurIPS 2024 preprint) and related EDFL/ISR/B2T methodology.
There doesn't seem to be a paper by that title, preprint or actual neurips publication. There is https://arxiv.org/abs/2507.11768, with a different title, and contains lots of inconsistencies with regards to the model. For example, from the appendix:
> All experiments used the OpenAI API with the following configuration:
> • Model: *text-davinci-002*
> • Temperature: 0 (deterministic)
> • Max tokens: 0 (only compute next-token probabilities)
> • Logprobs: 1 (return top token log probability)
> • Rate limiting: 10 concurrent requests maximum
> • Retry logic: Exponential backoff with maximum 3 retries
That model is not remotely appropriate for these experiments and was deprecated in 2023.
I'd suggest anyone excited by this attempt to run the codebase on github and take a close look at the paper.
Re: Hallucination Risk Calculator
#13Of course this is the risk, not the proof. High risk answers can be correct, low ones can still be partly hallucinated. And then there is the factor of shit-in-shit-out training data. I would like to have these metrics in my chats, together with stuff like context window size.
I just want a badge that says "ai-generated" for content thats likely ai slop on LinkedIn, Reddit, X etc.
Where's the boundary, though? If someone generated slop but edits every sentence replacing at least half the words and ensuring it is in their voice consistently, does it still need the badge? If only one word is replaced but it corrected the hallucination and is otherwise reviewed for approval? If dice are rolled and the x'th word of the y'th sentence chosen for replacement?
I don't justify starting with slop in my own writings but I don't know whether you could even reliably label it appropriately. Even more so, it would be a shame to see genuinely human writing mischaracterized as genAI, especially in a public forum like LinkedIn.
Re: Hallucination Risk Calculator
#14Earlier quoted context omitted.
I just want a badge that says "ai-generated" for content thats likely ai slop on LinkedIn, Reddit, X etc.
Where's the boundary, though? If someone generated slop but edits every sentence replacing at least half the words and ensuring it is in their voice consistently, does it still need the badge? If only one word is replaced but it corrected the hallucination and is otherwise reviewed for approval? If dice are rolled and the x'th word of the y'th sentence chosen for replacement? I don't justify starting with slop in my…
At work, we had video content with "ai generated" now shown in an eponymous Teams channel. Now no one viewing the video knows if just the image, or the project logo, and/or the voiceover was AI generated but is very confused on that upon seeing the label.
Skip the labels. Photoshop and "the trainee did it" existed for 38 years already, and respectively for many more years now, and have about the same reliability.
Re: Hallucination Risk Calculator
#15This topic is interesting, but the repo and paper have a lot of inconsistencies that make me think this work is hiding behind lots of dense notation and language. For one, the repo states: > This implementation follows the framework from the paper “Compression Failure in LLMs: Bayesian in Expectation, Not in Realization” (NeurIPS 2024 preprint) and related EDFL/ISR/B2T methodology. There doesn't seem to be a paper by…
It's telling that neither the repo nor the linked paper have a single empirical demonstration of the ability to predict hallucination. Let's see a few prompts and responses! Instead, all I see is a lot of handwavy philosophical pseudo-math, like using Kolmogorov complexity and Solomonoff induction, two poster children of abstract concepts that are inherently not computable, as explicit algorithmic objectives.
Re: Hallucination Risk Calculator
#16Earlier quoted context omitted.
I just want a badge that says "ai-generated" for content thats likely ai slop on LinkedIn, Reddit, X etc.
Where's the boundary, though? If someone generated slop but edits every sentence replacing at least half the words and ensuring it is in their voice consistently, does it still need the badge? If only one word is replaced but it corrected the hallucination and is otherwise reviewed for approval? If dice are rolled and the x'th word of the y'th sentence chosen for replacement? I don't justify starting with slop in my…
Then make it configurable per user, so people that dont want it can turn it off. Regarding the boundary idk but I still would rather know the likelihood that the content was AI generated (especially if it's high) than not.
Re: Hallucination Risk Calculator
#17Earlier quoted context omitted.
Where's the boundary, though? If someone generated slop but edits every sentence replacing at least half the words and ensuring it is in their voice consistently, does it still need the badge? If only one word is replaced but it corrected the hallucination and is otherwise reviewed for approval? If dice are rolled and the x'th word of the y'th sentence chosen for replacement? I don't justify starting with slop in my…
At work, we had video content with "ai generated" now shown in an eponymous Teams channel. Now no one viewing the video knows if just the image, or the project logo, and/or the voiceover was AI generated but is very confused on that upon seeing the label. Skip the labels. Photoshop and "the trainee did it" existed for 38 years already, and respectively for many more years now, and have about the same reliability.
Agreed. I think we'll see a further strengthening of reputation as a signal over any isolated statement or marketing.
Re: Hallucination Risk Calculator
#18Earlier quoted context omitted.
Where's the boundary, though? If someone generated slop but edits every sentence replacing at least half the words and ensuring it is in their voice consistently, does it still need the badge? If only one word is replaced but it corrected the hallucination and is otherwise reviewed for approval? If dice are rolled and the x'th word of the y'th sentence chosen for replacement? I don't justify starting with slop in my…
At work, we had video content with "ai generated" now shown in an eponymous Teams channel. Now no one viewing the video knows if just the image, or the project logo, and/or the voiceover was AI generated but is very confused on that upon seeing the label. Skip the labels. Photoshop and "the trainee did it" existed for 38 years already, and respectively for many more years now, and have about the same reliability.
That's telling me the label was over applied, not that the label itself was not important. I'd love something that just tells me the likelihood that a text post was AI generated to start (like on LinkedIn, for eg).
Re: Hallucination Risk Calculator
#19From the paper abstract, > (4) we derive the optimal chain-of-thought length as [..math..] with explicit constants I know we probably have to dive into math and abandon metaphor and analogy, but the whole structure of a claim like this just strikes me as bizarre. Chain-of-thought always makes me think of that old joke. Alexander the great was a great general. Great generals are forewarned. Forewarned is forearmed. Fo…
Yesterday I used ChatGPT to transform a csv file. Move around a couple of columns, add a few new ones. Very large file.
It got them all right. Except when I really looked through the data, for 3 of the excel cells, it clearly just made up new numbers. I found the first one by accident, the remaining two took longer than it would have taken to modify the file from scratch myself.
Watching my coworkers blindly trust output like this is concerning.
Re: Hallucination Risk Calculator
#20This seems less accurate than `return 1.0`
Using the unboundedly unreliable systems to evaluate reliability is just a bad premise.