Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I'm absolutely right

absolutelyright.lol

Post reply on HN
Print view View original

191–200 of 277 posts

Re: I'm absolutely right

#191

Post by Uehreka »

teucris wrote: post #174
Uehreka wrote: post #159

Earlier quoted context omitted.

The human stochastic parrots (GP, not you) spouting these 2023 talking points really need to update their weights. I’m guessing this way of thinking has a stickiness because thinking of an LLM as “just a fancy markov chain” makes them feel less threatening to some people (we’re past the point where it could be good faith reasoning). Like, I hear people say things like that (or that coding agents can only do web devel…

I think there’s a bit of parroting going around but LLMs are predictive and there’s a lot you can inuit a lot about how they behave just on that fact alone. Sure, calling it “token” prediction is oversimplifying things, but stating that, by their nature, LLMs are guessing at the next most likely thing in the scenario (next data structure needing to be coded up, next step in a process, next concept to cover in a parag…

Honestly, I think the best way to reason about LLM behavior is to abandon any sort of white-box mental model (where you start from things you “know” about their internal mechanisms). Treat them as a black box, observe their behavior in many situations and over a long period of time, draw conclusions from the patterns you observe and test if your conclusions have predictive weight.

Of course, if someone is predisposed to incuriosity about LLMs and refuses to use them, they won’t be able to participate in that approach. However I don’t think there’s an alternative.

Re: I'm absolutely right

#192

Post by Szpadel »

trjordan wrote: post #41

OK, so I love this, because we all recognize it. It's not fully just a tic of language, though. Responses that start off with "You're right!" are alignment mechanisms. The LLM, with its single-token prediction approach, follows up with a suggestion that much more closely follows the user's desires, instead of latching onto it's own previous approach. The other tic I love is "Actually, that's not right." That happens…

exactly!

People bless gpt-5 for not doing exactly this and in my testing with it in copilot I had lot of cases where it tried to do wrong thing (execute come messed up in context compaction build command) and I couldn't steer it to do ANYTHING else. It constantly tried to execute it as response any my message (I tries many common steerability tricks, (important, , just asking, yelling etc) nothing worked.

the same think when I tried to do socratic coder prompting, I wanted to finish and generate spec, but he didn't agree and kept asking nonsensical at this point questions

Re: I'm absolutely right

#193

Post by derefr »

adastra22 wrote: post #156
derefr wrote: post #128

Earlier quoted context omitted.

I think this poster is suggesting that, rather than "thinking" (messages emitted for oneself as audience) as a discrete step taken before "responding", the model should be trained to, during the response, tag certain sections with tokens indicating that the following token-stream until the matching tag is meant to be visibility-hidden from the client. Less "independent work before coming to the meeting", more "mumbli…

Doesn’t need training. Just don’t show it. Can be implemented client side.

Just don't show... what? The specific exact text "You're absolutely right!"?

That heuristic wouldn't even survive the random fluctuations in how the model says it (it doesn't always say "absolutely"; the punctuation it uses is random; etc); let alone speaking to the model in another language, or challenging the model in the context of it roleplaying a character or having been otherwise prompted to use some other personality / manner of speech (where it still does emit this kind of "self-reminder" text, but using different words that cohere with the set personality.)

The point of teaching a model to emit inline sequences, would be to allow the model to arbitrarily "mumble" (say things for its own benefit, that it knows would annoy people if spoken aloud), not just to "mumble" this one single thing.

Also, a frontend heuristic implies a specific frontend. I.e. it only applies to hosted-proprietary-model services that have a B2C chat frontend product offering tuned to the needs of their model (i.e. effectively just ChatGPT and Claude.) The text-that-should-be-mumbled wouldn't be tagged in any way if you call the same hosted-proprietary-model service through its API (so nobody building bots/agents on these platforms would benefit from the filtering.)

In contrast, if one of the hosted-proprietary-model chat services trained their model to tag its mumbles somehow in the response stream, then this would define an effective de-facto microformat for such mumbles — allowing any client (agent or frontend) consuming the conversation message stream through the API to have a known rule to pick out and hide arbitrary mumbles from the text (while still being able to make them visible to the user if the user desires, unlike if they were filtered out at the "business layer" [inference-host framework] level.)

And if general-purpose frameworks and clients began supporting that microformat, then other hosted-proprietary-model services — and orgs training open models — would see that the general-purpose frameworks/clients have this support, and so would seek to be compatible with that support, basically by aping the format the first mumbling hosted-proprietary-model emits.

(This is, in fact, exactly what already happened for the de-facto microformat that is OpenAI's reasoning-model explicit pre-response-message thinking-message format, i.e. the {"content_type": "thoughts", "thoughts": [{"summary": "...", "content": "..."}]} format.)

Re: I'm absolutely right

#194

Post by libraryofbabel »

Uehreka wrote: post #159
libraryofbabel wrote: post #145

Earlier quoted context omitted.

> The LLM, with its single-token prediction approach, follows up with a suggestion that much more closely follows the user's desires, instead of latching onto it's own previous approach. Maybe? How would we test that one way or the other? If there’s one thing I’ve learned in the last few years, it’s that reasoning from “well LLMs are based on next-token prediction, therefore ” is a trap. The relationship between the…

The human stochastic parrots (GP, not you) spouting these 2023 talking points really need to update their weights. I’m guessing this way of thinking has a stickiness because thinking of an LLM as “just a fancy markov chain” makes them feel less threatening to some people (we’re past the point where it could be good faith reasoning). Like, I hear people say things like that (or that coding agents can only do web devel…

You’re being downvoted, perhaps because your tone is a little harsh, but you’re not wrong: people really are still making versions of the “stochastic parrots” argument. It comes up again and again, on hacker news and elsewhere. And yet a few months ago an LLM got gold on the Mathematical Olympiad. “Stochastic parrots” just isn’t a useful metaphor anymore.

I find AI hype as annoying as anyone, and LLMs do have all sorts of failure modes, some of which are related to how they are trained. But at this point they are doing things that many people (including me) would have flatly denied was possible with this architecture 3 years ago during the initial ChatGPT hype. When the facts change we need to change our opinions, and like you say, reckon anew with the thing that’s sitting in front of us.

Re: I'm absolutely right

#195

Post by ZaoLahma »

kuschku wrote: post #80
ZaoLahma wrote: post #70

Earlier quoted context omitted.

Using your potato peeler example: If my potato peeler told me "Why bother? Order pizza instead." I'd be obese. An LLM can directly influence your willingness to pursue an idea by how it responds to it. Interest and excitement, even if simulated, is more likely to make you pursue the idea than "ok cool".

> If my potato peeler told me "Why bother? Order pizza instead." I'd be obese. But why do you let yourself be influenced so much by others, or in this case, random filler words from mindless machines? You should listen to your own feelings, desires, and wishes, not anything or anyone else. Try to find the motivation inside of you, try to have the conversation with yourself instead of with ChatGPT. And if someone tell…

It’s not “by others”. It’s by circumstance.

It’s like any other tool. If I wanted to chop wood and noticed how my axe had gone dull, the likelihood of me going “ah f*ck it” and instead go fishing increases dramatically. I want to chop wood. I don’t want to go to the neighbor and borrow his axe, or sharpen my axe and then chop wood.

That’s what has happened with ChatGPT in a sense - it has gone dull. I know it used to work “better” and the way that it works now doesn’t resonate with me in the same way, so I’m less likely to pursue work that I would want to use ChatGPT as an extrinsic motivator for.

Of course if the intrinsic motivation is large enough I wouldn’t let a tool make the decision for me. If it’s mid October and the temperature is barely above freezing and I have no wood, I’ll gnaw through it with my teeth if necessary. I’ll go full beaver. But in early September when it’s 25C outside on a Friday? If the axe isn’t perfect, I’ll have a beer and go fishing.

Re: I'm absolutely right

#196

Post by libraryofbabel »

Uehreka wrote: post #191
teucris wrote: post #174

Earlier quoted context omitted.

I think there’s a bit of parroting going around but LLMs are predictive and there’s a lot you can inuit a lot about how they behave just on that fact alone. Sure, calling it “token” prediction is oversimplifying things, but stating that, by their nature, LLMs are guessing at the next most likely thing in the scenario (next data structure needing to be coded up, next step in a process, next concept to cover in a parag…

Honestly, I think the best way to reason about LLM behavior is to abandon any sort of white-box mental model (where you start from things you “know” about their internal mechanisms). Treat them as a black box, observe their behavior in many situations and over a long period of time, draw conclusions from the patterns you observe and test if your conclusions have predictive weight. Of course, if someone is predisposed…

This is precisely what I recommend to people starting out with LLMs: do not start with the architecture, start with their behavior - use them for a while as a black box and then circle back and learn about transformers and cross entropy loss functions and whatever. Bottom-up approaches to learning work well in other areas of computing, but not this - there is nothing in the architecture to suggest the emergent behavior that we see.

Re: I'm absolutely right

#197

Post by octo888 »

ZaoLahma wrote: post #34
nicce wrote: post #16

Earlier quoted context omitted.

It is a tactic. OpenAI is changing the tone of ChatGPT if you use casual language, for example. Sometimes even the dialect. They try to be sympathetic and supportive, even when they should not. They fight for the user attention and keeping them on their platform, just like social media platforms. Correctness is secondary, user satisfaction is primary.

I find the GPT-5 model having turned the friendliness way, way down. Topics that previously would have rendered long and (usefully) engaging conversations are now met with an "ok cool" kind of response. I get it - we don't want LLMs to be reinforces of bad ideas, but sometimes you need a little positivity to get past a mental barrier and do something that you want to do, even if what you want to do logically doesn't…

This reads like a submarine ad lol. Especially the second paragraph

Re: I'm absolutely right

#198

Post by genewitch »

eaf wrote: post #118
calflegal wrote: post #24

As a joke I built https://idk-ask-ai.com/

Recently a new philosophy of parenting has been emerging, which can be termed “vibe parenting” and describes a novel method for the individual parent to circumvent an inability to answer the sporadic yet profound questions their children raise by directing them to ask ChatGPT. https://x.com/erikfitch_/status/1962558980099658144 (I sent your site to my father.)

My parents, 40 years ago, would say "look it up", either in the dictionary, or the 1959 encyclopedia set we had. With my kids i never told them to look something up in the literal dictionary, but i would tell them to look at wikipedia or "google it". Not about profound questions, though; although a definition of "profound questions" might jog a memory. We do look things up in an etymology dictionary (i have 5 or 6) sometimes, though.

I am not sure why my parents constantly told me to look things up in a dictionary.

Rarely, but it did happen, we'd have to take a trip to the library to look something up. Now, instead of digging in a card catalog or asking a librarian, and then thumbing through reference books, i can ask an LLM to see if there's even information plausibly available before dedicating any more time to "looking something up."

As i've been saying lately, i use copilot to see if my memory is failing.

Re: I'm absolutely right

#199

Post by data-ottawa »

adastra22 wrote: post #155
nojs wrote: post #61

Earlier quoted context omitted.

Yeah, I figure this is also why it often says “Ah, I found the problem! Let me check the …”. It hasn’t found the problem, but it’s more likely to continue with the solution if you jam that string in there.

We don’t know how Claude code is internally implemented. I would not be surprised at all if they literally inject that string as an alternative context and then go with the higher probability output, or if RLHF was structured in that way and so it always generates the same text.

Very likely RLHF, based only on how strongly aligned open models repeatedly reference a "policy" despite there being none in the system prompt.

I would assume that priming the model to add these tokens ends up with better autocomplete as mentioned above.

Re: I'm absolutely right

#200

Post by wrs »

kevinventullo wrote: post #186
wrs wrote: post #133

Earlier quoted context omitted.

Long ago (I first remember doing it in about 1985 with the original Mac watch cursor) this was the standard way to do spinners: somewhere in your processing you incremented the spinner. It was hard to put the increments in the right places to keep the spinner going even when progress was happening, and nearly impossible to make it animate smoothly. This technique is even harder to get right when the processing in que…

What’s funny is the jerky animation actually communicates so much more than the smooth animation.

Yeah, same with progress bars — especially the Windows Installer progress bar that goes backwards when it’s backing out after a failure!

Of course, progress bars based on increments have a whole other failure mode, the eternally 99% progress bar…

Post reply on HN