An LLM is a lossy encyclopedia • Hacker News Board

Re: An LLM is a lossy encyclopedia

#361

Post by simonw » Sat, Sep 06, 2025, 11:35 AM UTC

slacktivism123 wrote: post #360

simonw wrote: post #359

Earlier quoted context omitted.

The reason I shout "weird and unintuitive" from the rooftops is that no LLM vendor will ever describe their weird and unintuitive products that way.

Describing the commercial offerings as "weird and unintuitive" is a weak criticism palatable to corporate comms teams. It suggests a fault in the user ("you're holding it wrong") rather than deficiencies inherent to LLM architecture. No amount of marketing can fix the lethal trifecta or the hallucination problem, can it? https://www.anthropic.com/solutions/code-modernization : Generate dependency graphs, identify dea…

OpenAI actually put out an interesting paper on addressing hallucination yesterday, but I've not spent enough time with it to judge how credible it is: https://openai.com/index/why-language-models-hallucinate/

I don't particularly care how these companies market their software - what I care about is figuring out what these things can actually do and what they're genuinely useful for, then helping other people use them in as productive a way as possible given their inherent flaws.

Re: An LLM is a lossy encyclopedia

#362

Post by dncornholio » Tue, Sep 09, 2025, 12:25 PM UTC

el_benhameen wrote: post #188

dncornholio wrote: post #143

Earlier quoted context omitted.

Using a LLM for medical research is just as dangerous as Googling it. Always ask your doctors!

I don’t disagree that you should use your doctor as your primary source for medical decision making, but I also think this is kind of an unrealistic take. I should also say that I’m not an AI hype bro. I think we’re a long ways off from true functional AGI and robot doctors. I have good insurance and have a primary care doctor with whom I have good rapport. But I can’t talk to her every time I have a medical question…

I can e-mail my doctor and have a response within 2 days. He is not working alone, but has multiple assistants working. This is a normal doctors office that everyone is required to have in the Netherlands.

E-mails and communication is completely free of charge.

We all know that Google and LLM's are not the answer for your medical questions but that they cause fear and stress instead.

Re: An LLM is a lossy encyclopedia

#363

Post by dncornholio » Tue, Sep 09, 2025, 12:30 PM UTC

el_benhameen wrote: post #313

lurking_swe wrote: post #303

Earlier quoted context omitted.

I live in the U.S. and my doctor is very responsive on MyChart. A few times a year i’ll send a message and I almost always get a reply within a day! From my PCP directly, or from her assistant. I’d encourage you to find another doctor.

My doctor is usually pretty good at responding to messages too, but there’s still a difference between a high-certainty/high-latency reply and a medium-certainty/low-latency reply. With the llm I can ask quick follow ups or provide clarification in a way that allows me to narrow in on a solution without feeling like I’m wasting someone else’s time. But yes, if it’s bleeding, hurting, or growing, I’m definitely going…

You are NOT wasting someone else's time, they get paid to do just that, answer questions.. Plus it's your fucking health dude

Re: An LLM is a lossy encyclopedia

#364

Post by jpcompartir » Tue, Sep 09, 2025, 2:10 PM UTC

ACCount37 wrote: post #219

jpcompartir wrote: post #168

Earlier quoted context omitted.

You seem to be responding to a strawman, and assuming I think something I don't think. As of today, 'bad' generations early in the sequence still do tend towards responses that are distant to the ideal response. This is testable/verifiable by pre-filling responses, which I'd advise you to experiment with for yourself. 'Bad' generations early in the output sequence are somewhat mitigatable by injecting self-reflection…

With better reasoning training, the models mitigate more and more of that entirely by themselves. They "diverge into a ditch" less, and "converge towards the right answer" more. They are able to use more and more test-time compute effectively. They bring their own supply of "wait". OpenAI's in-house reasoning training is probably best in class, but even lesser naive implementations go a long way.

Assuming you've read OpenAI's paper released this week?

https://cdn.openai.com/pdf/d04913be-3f6f-4d2b-b283-ff432ef4a...

They attribute these 'compression artefacts' to pre-training, they also reference the original snowballing paper: How Language Model Hallucinations Can Snowball: https://arxiv.org/pdf/2305.13534

They further state that reasoning is no panacea. W hilst you did say: "the models mitigate more and more"

You were replying to my comment which said:

"'Bad' generations early in the output sequence are somewhat mitigatable by injecting self-reflection tokens like 'wait', or with more sophisticated test-time compute techniques."

So our statements there are logically compatible, i.e. you didn't make a statement that contradicts what I said.

"Our error analysis is general yet has specific implications for hallucination. It applies broadly, including to reasoning and search-and-retrieval language models, and the analysis does not rely on properties of next-word prediction or Transformer-based neural networks."

"Search (and reasoning) are not panaceas. A number of studies have shown how language models augmented with search or Retrieval-Augmented Generation (RAG) reduce hallucinations (Lewis et al., 2020; Shuster et al., 2021; Nakano et al., 2021; Zhang and Zhang, 2025). However, Observation 1 holds for arbitrary language models, including those with RAG. In particular, the binary grading system itself still rewards guessing whenever search fails to yield a confident answer. Moreover, search may not help with miscalculations such as in the letter-counting example, or other intrinsic hallucinations"

Re: An LLM is a lossy encyclopedia

#365

Post by nexoft » Wed, Sep 10, 2025, 9:38 AM UTC

I don't understand these low effort, superficial takes on complex topics from non-specialists trying to get visibility.\ This is literally 101 neural network/deep learning fact you learn when you are introduced to LLMs : it is a non-lossless compression of large dataset.

[Anredj Karpathy first minutes of Intro to Large Language Models / 4Min20](https://youtu.be/zjkBMFhNj_g?t=257)

Training is a compression of a huge chunk of the internet.

The replies and interactions looks like a bunch of agentic models discussing with each others. Nobody is real.