This is such a bizarre bug-ish thing and while Claude loves the "You're absolutely right!" trope, it's downright haunting how stuff like ChatGPT has become my own personal fan club. It's like a Jim Jones factory.
I'm absolutely right
171–180 of 277 posts
Re: I'm absolutely right
#172How many times did it say "Looking at the _, I can see the problem"
Re: I'm absolutely right
#173I was just thinking about how LLM agents are both unabashedly confident (Perfect, this is now production-ready!) and sycophantic when contradicted (You're absolutely right, it's not at all production-ready!)
It's a weird combination and sometimes pretty annoying. But I'm sure it's preferable over "confidently wrong and doubling down".
Re: I'm absolutely right
#174Earlier quoted context omitted.
> The LLM, with its single-token prediction approach, follows up with a suggestion that much more closely follows the user's desires, instead of latching onto it's own previous approach. Maybe? How would we test that one way or the other? If there’s one thing I’ve learned in the last few years, it’s that reasoning from “well LLMs are based on next-token prediction, therefore ” is a trap. The relationship between the…
The human stochastic parrots (GP, not you) spouting these 2023 talking points really need to update their weights. I’m guessing this way of thinking has a stickiness because thinking of an LLM as “just a fancy markov chain” makes them feel less threatening to some people (we’re past the point where it could be good faith reasoning). Like, I hear people say things like that (or that coding agents can only do web devel…
I think there’s a bit of parroting going around but LLMs are predictive and there’s a lot you can inuit a lot about how they behave just on that fact alone. Sure, calling it “token” prediction is oversimplifying things, but stating that, by their nature, LLMs are guessing at the next most likely thing in the scenario (next data structure needing to be coded up, next step in a process, next concept to cover in a paragraph, etc.) is a very useful mental model.
Re: I'm absolutely right
#175Is it too much to ask for an AI that says "you're absolutely wrong," followed by a Stack Overflow-style shakedown?
Re: I'm absolutely right
#176Sycophancy As A Service
Re: I'm absolutely right
#177Earlier quoted context omitted.
The human stochastic parrots (GP, not you) spouting these 2023 talking points really need to update their weights. I’m guessing this way of thinking has a stickiness because thinking of an LLM as “just a fancy markov chain” makes them feel less threatening to some people (we’re past the point where it could be good faith reasoning). Like, I hear people say things like that (or that coding agents can only do web devel…
I think there’s a bit of parroting going around but LLMs are predictive and there’s a lot you can inuit a lot about how they behave just on that fact alone. Sure, calling it “token” prediction is oversimplifying things, but stating that, by their nature, LLMs are guessing at the next most likely thing in the scenario (next data structure needing to be coded up, next step in a process, next concept to cover in a parag…
I would challenge the utility of this mental model as again they're not simply tracing a "most likely" path unless your sampling methods are trivially greedy. I don't know of a better way to model it, and I promise I'm not trying to be anal here
Re: I'm absolutely right
#178As I opened the website, the “16” changed to “17”. This looked interesting, as if the data were being updated live just as I loaded the page. Alas, a refresh (and quick check in the Developer Tools) reveals it’s fake and always does the transition. It’s a cool effect, but feels like a dirty trick.
Sorry if that felt dirty - I thought about it as a signal that the data is live (it is!).
Might make more sense to start at zero and then rapidly scale to the current number? To indicate fresh data is being loaded in without making it look like the reader happened to catch a new occurrence in real-time.
Re: I'm absolutely right
#179I was just thinking about how LLM agents are both unabashedly confident (Perfect, this is now production-ready!) and sycophantic when contradicted (You're absolutely right, it's not at all production-ready!) It's a weird combination and sometimes pretty annoying. But I'm sure it's preferable over "confidently wrong and doubling down".
we cannot claim to have built human level intelligence until "confidently wrong and doubling down" is the default.