Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How to stop Claude from saying load-bearing

jola.dev

Post reply on HN
Print view View original

61–70 of 650 posts

Re: How to stop Claude from saying load-bearing

#61

Post by bunderbunder »

Myrmornis wrote: post #54
bigfishrunning wrote: post #42

Earlier quoted context omitted.

but you don't see "load bearing" nearly as often in prose written by people, so it's not some irreplaceable phrase. It's just a token with a weirdly high likelihood in a lot of cases (given how Claude works, this kind of thing is bound to happen)

You don't think it's possible that an LLM's internal machinery could decide that an underused-by-humans word should be used more frequently in output than it sees in input because it maps cleanly onto a frequently needed semantic? I think that's possible

It sounds like you are trying to understand LLM behavior using a mental model that inaccurately personifies the stochastic parrot.

A more parsimonious explanation is that this term got more-or-less randomly boosted by the reinforcement learning loop because there was nothing in the training data to discourage its use.

Re: How to stop Claude from saying load-bearing

#62

Post by Myrmornis »

bunderbunder wrote: post #61
Myrmornis wrote: post #54

Earlier quoted context omitted.

You don't think it's possible that an LLM's internal machinery could decide that an underused-by-humans word should be used more frequently in output than it sees in input because it maps cleanly onto a frequently needed semantic? I think that's possible

It sounds like you are trying to understand LLM behavior using a mental model that inaccurately personifies the stochastic parrot. A more parsimonious explanation is that this term got more-or-less randomly boosted by the reinforcement learning loop because there was nothing in the training data to discourage its use.

Ah right, you don't like AI and don't care to understand how it works.

Re: How to stop Claude from saying load-bearing

#63

Post by bunderbunder »

Foskya wrote: post #56
bunderbunder wrote: post #18

I recently started using caveman, and it’s been great. It doesn’t just cut down on overuse of specific terms; it cuts down on time spent digesting slop in general. https://github.com/JuliusBrussee/caveman

I love it. It also saves you tokens and it has been linked with more accuracy.

The token saving is oversold, from what I can tell so far. These days output tokens are just the tip of the iceberg.

If anything the real value is it saves my brain from going into power saving mode by lunchtime because I haven’t spent the day reading pages of output when a sentence or two would do.

Re: How to stop Claude from saying load-bearing

#64

Post by anon373839 »

p-e-w wrote: post #15
pocketarc wrote: post #11

In the olden days, I enjoyed Opus 3 because it was easy to have it sound way more human than GPT. Nowadays, with the focus on agentic use and coding, it seems models have all been RLHF’d to death, it’s so incredibly hard to have them write in a different voice than their default. I put together a skill to review its writing and have it edit its own output (e.g. code comments), which does make a difference, but isn’t…

> Nowadays, with the focus on agentic use and coding, it seems models have all been RLHF’d to death I don’t get it. If nobody likes this writing style, how can it be the result of human feedback? Something else is going on.

Because LLMs are pattern-extenders that have nothing to say. The training overfitted to the grace notes in good writing. And since LLMs can’t wield language with purpose or experience the feeling of the words, they use these devices arbitrarily.

I think this is the same flaw as coding agents seeing in every problem the call for a “smoke test” or the use of some unnecessary design pattern. The truest part of AI is the A.

Re: How to stop Claude from saying load-bearing

#65

Post by bunderbunder »

Myrmornis wrote: post #62
bunderbunder wrote: post #61

Earlier quoted context omitted.

It sounds like you are trying to understand LLM behavior using a mental model that inaccurately personifies the stochastic parrot. A more parsimonious explanation is that this term got more-or-less randomly boosted by the reinforcement learning loop because there was nothing in the training data to discourage its use.

Ah right, you don't like AI and don't care to understand how it works.

I’ve been working in AI - and specifically NLP - since 2003. I am no stranger to how weird quirks can sneak into overparametrized models, nor am I a stranger to how good humans can be at inferring meaning where there is none in specific language model behaviors. So, yeah, I am inclined to assume non-teleological causes are more parsimonious than inferring the presence of a strange loop, because that continues to be the winning bet. Even for generative LLMs.

Re: How to stop Claude from saying load-bearing

#67

Post by bigfishrunning »

Myrmornis wrote: post #62
bunderbunder wrote: post #61

Earlier quoted context omitted.

It sounds like you are trying to understand LLM behavior using a mental model that inaccurately personifies the stochastic parrot. A more parsimonious explanation is that this term got more-or-less randomly boosted by the reinforcement learning loop because there was nothing in the training data to discourage its use.

Ah right, you don't like AI and don't care to understand how it works.

Ah right, so you like AI and don't care to understand how it works.

It doesn't "decide" anything or "need" any semantic. It derives the likelihood of the token, and "bearing" is likely to come after "load".

Re: How to stop Claude from saying load-bearing

#68

Post by retsibsi »

dfc wrote: post #53
graemep wrote: post #45

Earlier quoted context omitted.

Claude does at least use the British English version of the phrase to me - not sure whether its picking up a language setting or reacting to my spelling etc. The American version does sound odd over hear.

What's the difference between the two usages?

"Belt and braces" (UK) vs. "belt and suspenders" (US). I'm pretty sure the phrases have the same meaning, they just use a different word to refer to the thing that holds pants|trousers up.

Re: How to stop Claude from saying load-bearing

#69

Post by Myrmornis »

Retr0id wrote: post #51
Myrmornis wrote: post #35

I like to think that the reason it's so noticable is that Claude has recognized some important semantics that we ourselves lack a good word for or at least under-appreciate. What term is used in English (or other languages) with the same meaning as claude's "load-bearing"? operative? key? critical? decisive? The honest conclusion is that none of those are as good as "load-bearing". And yet the concept being referred…

> The honest conclusion I think you've been reading too much claude output! "Load bearing" is cromulent verbiage and can be used in many scenarios - so claude does. But variety is important too, and there are more specific alternatives that can be used in most situations. Any word becomes a bad choice if you've used it 10 times in the last chapter.

You don't think me using "honest" there might have been a tiny bit of (on-topic, and therefore appropriate) trolling?

Re: How to stop Claude from saying load-bearing

#70

Post by Myrmornis »

bigfishrunning wrote: post #67
Myrmornis wrote: post #62

Earlier quoted context omitted.

Ah right, you don't like AI and don't care to understand how it works.

Ah right, so you like AI and don't care to understand how it works. It doesn't "decide" anything or "need" any semantic. It derives the likelihood of the token, and "bearing" is likely to come after "load".

Sure but the question is why "load" after X?
Post reply on HN