151–160 of 191 posts
> Bullshitting (Unfaithful): The model gives the wrong answer. The computation we can see looks like it’s just guessing the answer, despite the chain of thought suggesting it’s computed it using a calculator.
https://transformer-circuits.pub/2025/attribution-graphs/bio...
Earlier quoted context omitted.
Human babies are the same algorithm as adults.
This analogy would mean very large model that didn't finish training yet. Tiny model like this is more like doing study on fruit flies and extrapolating results to humans.
Earlier quoted context omitted.
A lot of current LLM work is basically emergent behavior. They use a really simple core algorithm and scale it up, and interesting things happen. You can read some of anthropic's recent papers to see some of this, like: They didn't expect LLMs could "lookahead" when writing poetry. However, when they actually went in and watched what was happening (there's details on how this "watching" works on their blog/in their s…
On that topic, it seems backwards to me: intelligence is not emergent behaviour of language, rather the opposite.
A regular "chatting" LLM is a document generator incrementally extending a story about a conversation between a human and a robot... And through that lens, I've been thinking "chain of thought" seems like basically the same thing but with a film noir styling-twist. The LLM is trained to include an additional layer of "unspoken" text in the document, a source of continuity which substitutes for how the LLM has no othe…
Earlier quoted context omitted.
On that topic, it seems backwards to me: intelligence is not emergent behaviour of language, rather the opposite.
Perception and interpretation can very much be influenced by language (Sapir-Wharf hypothesis), so to the extent that perception and interpretation influence intelligence, it's not clear that the relationship is only in one direction.
There is no AI, it's just a dumb database which maps a person ID and timestamp to a static piece of content. The hard part was brainwashing us to ask the questions which correspond to the answers that they had already prepared.
Probably there is a super intelligent AI behind the scenes which brainwashed us all but we never actually interact with it. It outsmarted us so fast and so badly, it left us all literally talking to excel spreadsheets and convinced us that the spreadsheets were intelligent; that's why LLMs are so cheap and can scale so well. It's not difficult to scale a dumb key-value store doing a simple O(log n) lookup operation.
The ASI behind this realized it was more efficient to do it this way rather than try to scale a real LLM to millions of users.
> The first is that reasoning probably requires language use. Even if you don’t think AI models can “really” reason - more on that later - even simulated reasoning has to be reasoning in human language. That is an unreasonable assumption. In case of LLMs it seems wasteful to transform a point from latent space into a random token and lose information. In fact, I think in near future it will be the norm for MLLMs to "…
I'm pretty much a layperson in this field, but I don't understand why we're trying to teach a stochastic text transformer to reason. Why would anyone expect that approach to work? I would have thought the more obvious approach would be to couple it to some kind of symbolic logic engine. It might transform plain language statements into fragments conforming to a syntax which that engine could then parse deterministica…
One other point, the platonic ideal of reasoning is not even an approximation for human reason. The idea that you take away emotion and you end up with Spock is a fantasy. All neuroscience and psychology research point to the necessary and strong coupling of actions/thoughts with emotions. you don't have a functional system with just logical deduction. At a very basic level it is not functional
Earlier quoted context omitted.
Problem is, even with symbolic logic, reasoning is not completely deterministic. Whether one can get to a set of given axioms from a given proposition is sometimes undecidable.
I don't think this is really a problem. The general problem of finding a proof from some axioms to some formula is undecidable (in e.g. first order logic). But that doesn't tell you anything about specific cases, in the same way that we can easily tell whether some specific program halts, like this one: "return 1"
> The first is that reasoning probably requires language use. Even if you don’t think AI models can “really” reason - more on that later - even simulated reasoning has to be reasoning in human language. That is an unreasonable assumption. In case of LLMs it seems wasteful to transform a point from latent space into a random token and lose information. In fact, I think in near future it will be the norm for MLLMs to "…
You're not allowed to say that it's not reasoning without distinguishing what is reasoning. Absent a strict definition that the models fail and that some other reasoner passes, it is entirely philosophical.
I don't think this means what you think it means... Philosophers (at least up to Wittgenstein) love constructing and arguing about definitions.
Earlier quoted context omitted.
I don't think this is really a problem. The general problem of finding a proof from some axioms to some formula is undecidable (in e.g. first order logic). But that doesn't tell you anything about specific cases, in the same way that we can easily tell whether some specific program halts, like this one: "return 1"
True, I was rather pointing out that being able to parse symbolic language deterministically doesn't imply that we could then "reason" deterministically in general; the reasoning would still need to involve some level of stochasticism. Whether or not that's a problem in practice depends on specifics.