221–230 of 268 posts
Earlier quoted context omitted.
Plus it's an open question if this is even a thing. Does consciousness consist of constructing actions beforehand, or of construction justifications afterward? Frankly, my opinion is that DNA is incredible at choose the most energy efficient/cheap option, and the cheaper option is definitely justifications afterward. I feel strengthened by psychological experiments where people are shown fake events involving them, w…
Sufficiently constrained post-hoc justifications are indistinguishable from explanations. Consciousness tries to make things up, it learns that people notice this, it then begins trying to construct justifications that won't be predictably called out as false. Eventually it learns how its unconscious operates, and how to interrogate it, and its post-hoc justifications, at least in the common cases, become reliable.
There's a logical "skip" between that and
>Eventually it learns how its unconscious operates, and how to interrogate it, and its post-hoc justifications, at least in the common cases, become reliable.
The brain constructs a narrative that won't be called out as false, one that provides social capital, makes one feel good about oneself, is consistent with all your other justifications, etc. It's only an assumption that this process would naturally converge on Truth, and considering it's massively-multiplayer chaos where brains coordinate their stories in complex ways, my assumption is that this would converge on *stability*, not truth.
Earlier quoted context omitted.
You can never ask why a model did a certain thing, or what it was "thinking" when it said something - just like you can't ask a human which neurons were firing when they had a certain thought. The information just isn't available at that level. You absolutely can have deep nuanced discussions with LLMs however, you just need to better understand their strengths and weaknesses.
You certainly can ask it what it was thinking, the problem is just that it's more likely to make up a plausible sounding fabrication than to say "I don't know" or "my reasoning is hidden for business reasons" (frontier models hide a lot of their chain of thought). Which is the fundamental problem with LLMs though, if the data doesn't exist or it's sparse they make things up.
Earlier quoted context omitted.
> While deterministic, there is not a fixed number of ways it can express itself, given that we can use settings like temperature to inject randomness into the output. You're missing the point, which is that no matter the process involved. The LLM can only ever output one of the tokens in its token vector. It can't invent a new symbol or character. It can't leave and go build a church. It has to output a little piece…
You're moving the goalpost. If the definition of intelligence is based on ability to "go build a church", then we've ruled out the vast majority of the animal kingdom from being labeled "intelligent". If you cannot be consistent in your definition of "intelligence", then you cannot have a reliable litmus test for it.
Either way, if you consider animals, LLMs are even more poorly positioned. They can do exactly none of the things my cat can do. An LLM can string together words, but if my cat is intelligent, it's clear that stringing together words is not synonymous with intelligence, since my cat can't do that.
Earlier quoted context omitted.
You say you can have increasingly nuanced discussions with stronger models. What I say is, when I asked Claude why he applied a certain change I didn't understand, and boy, it was a small change, he said he "reasoned from first principles" based on the code paths. But it didn't work, and when I asked, "Okay, describe the steps of your reasoning from first principles," it literally answered that it had just made it up…
Posts like this are meaningless without more context - the model you're using, the harness, the initial prompt and context. Fable is better than most staff engineers at my FAANG.
Earlier quoted context omitted.
You certainly can ask it what it was thinking, the problem is just that it's more likely to make up a plausible sounding fabrication than to say "I don't know" or "my reasoning is hidden for business reasons" (frontier models hide a lot of their chain of thought). Which is the fundamental problem with LLMs though, if the data doesn't exist or it's sparse they make things up.
Choosing plausible sounding fabrication over an admission of ignorance is not an uncommon modality among the human beings I interact with, so I'm not surprised this pattern is found in models trained on human interactions.
That's the whole problem with all of these discussions. It's whataboutism and "You're holding it wrong" allegations.
Earlier quoted context omitted.
You say you can have increasingly nuanced discussions with stronger models. What I say is, when I asked Claude why he applied a certain change I didn't understand, and boy, it was a small change, he said he "reasoned from first principles" based on the code paths. But it didn't work, and when I asked, "Okay, describe the steps of your reasoning from first principles," it literally answered that it had just made it up…
Posts like this are meaningless without more context - the model you're using, the harness, the initial prompt and context. Fable is better than most staff engineers at my FAANG.
Earlier quoted context omitted.
If you were working as a manager on a large project, how would you build a model? Something where your position requires you to have an overview of the project but not necessarily to actually write or review much code.
I am not able to find it now, but there was an amazing story recently from the 60s or 70s where an engineer was in exactly this position. His team was building a new, complex, ambitious operating system, but it was late and over budget and didn’t work. It nearly wrecked the company. He talks about hitting rock bottom and asking himself what went wrong, and one of the fathers of computing (can’t remember which) shouts…
As for your suggestion, understanding every line might have worked in the 70s but even pre-agentic modern coding it's not possible for any large project with dependencies even if you are directly contributing code yourself, so I'm not sure how useful your idea is.
There's definitely cases where you should have that aim - writing a low level maths or graphics library, for example. But most people are not doing that.
Earlier quoted context omitted.
claude is so slow for interactive use like described, do people just run it in low effort mode or what?
I multi-task. While I’m waiting for Claude, I either check email or work with a different instance of Claude on a second problem