251–260 of 650 posts
Lots of people have their own voice and tend to prefer certain phrases. This has been the case for a long time and is generally not a big issue. Now LLMs come along and they also have their own phrasing preferences. But now it's a problem because what used to be personal preferences of a single person that manifests in 5000 words per day from one person tops, is now the bias of a single model multiplied x10,000,000,0…
Lots of people have their own voice and tend to prefer certain phrases. This has been the case for a long time and is generally not a big issue. Now LLMs come along and they also have their own phrasing preferences. But now it's a problem because what used to be personal preferences of a single person that manifests in 5000 words per day from one person tops, is now the bias of a single model multiplied x10,000,000,0…
I think it might be even worse. LLMs seem to get tragically stuck on certain patterns. Maybe it's partly because a pile of weights essentially always starts from scratch in the same condition, but even within a single conversation, it will literally just latch onto words and repeat them incessantly, to the point where it becomes annoying. So for example, current Claude models love "honest". They are always producing…
I have a delightful time poisoning my company's AI system this way.
I invented my own word that sounds perfectly cromulent† to an ordinary person, and any brain that's read a book learns how to infer meaning from context, so it's not a problem.
When I get a e-mail response from a coworker using my special word incorrectly, then I know it's AI and I respond telling the coworker I don't know what that word means. Busted.
† It's not actual "cromulent," but any Simpsons fan or human brain will know what I mean.
This makes me wonder if the reason why agents love weird punctuation is because the labs run the base models through a RL training step that forces them to correct their grammar; but instead of rewriting short spliced sentences into long coherent sentences, they just learn to splice them together with punctuation that passes the automatic grammar checker.
It's not that it uses certain phrases, it's that it settles on predictable speech patterns and uses them incessantly. What's funny is that humans do this too, but we don't find it irritating; we just call it a speaking style. But when a machine does it, it drives us crazy. Very interesting psychological phenomenon there.
Fascinatingly, I'm now so allergic to certain LLM-phrases that I immediately noticed your use of Not X but Y in this comment. Maybe that was intentional, maybe not, but it's a funny illustration of how odd this language rabbit hole has been!
I don't feel as triggered LLM phrasing as people report here. At most, it feels like the same inane corporate jargon I've rolled my eyes at for my whole career. Perhaps it is amped up a bit, with too many forms of jargon multiplexed? It's a bit like when multilingual people code-switch too rapidly or even start to form some pidgin language. However, it is lacking the shared social context for this switching to be communicative. It's a bit more like spinning the dial on an old radio with random cuts between programming styles.
Stripped bare, I think What bugs me is the aggravated feeling that I am wading through word salad, and no longer being able to give the purveyor the benefit of the doubt. It was frustrating enough in the past, when it came from someone who was struggling to write or express themselves well. But now, it carries the implicit insult that they didn't even try, and it is constant and unrelenting.
So for me it's not the phrasing, it's that the phrases eventually don't add up. The meandering feels like a random walk. I get the same feeling from a lot of the egregious generated code I see in my day job. It's all superficial window dressing, but seems to miss the signature of an actual mind grappling with ideas and having intent to communicate.
It feels like we're trapped in some elaborate conceptual art piece, confronted by impenetrable symbolism. It invites nihilism but doesn't seem to actually reflect an artistic intent. The abyss gazes back...
I like to think that the reason it's so noticable is that Claude has recognized some important semantics that we ourselves lack a good word for or at least under-appreciate. What term is used in English (or other languages) with the same meaning as claude's "load-bearing"? operative? key? critical? decisive? The honest conclusion is that none of those are as good as "load-bearing". And yet the concept being referred…
but you don't see "load bearing" nearly as often in prose written by people, so it's not some irreplaceable phrase. It's just a token with a weirdly high likelihood in a lot of cases (given how Claude works, this kind of thing is bound to happen)
Unfortunately, we're starting to now.
Thanks to Claude.
Lots of people have their own voice and tend to prefer certain phrases. This has been the case for a long time and is generally not a big issue. Now LLMs come along and they also have their own phrasing preferences. But now it's a problem because what used to be personal preferences of a single person that manifests in 5000 words per day from one person tops, is now the bias of a single model multiplied x10,000,000,0…
/s
Earlier quoted context omitted.
I'm not claiming to know better than researchers. They do know how they work, and so does everyone else, except you I guess. The research goals were and still are clearly distinct from the business goals.
Understanding how transformers work does not mean understanding how they compose into the capabilities we observe. The former is concretely understood. The latter is an active area of research where no, we (in general, including you) do not understand how they work.
This isn't people merely annoyed with repetition. This is the majority of people realizing the limitations of LLMs. Why would researchers give a flying crap about the ignorance of the business world and the public?
Earlier quoted context omitted.
Yeah wow fascinating! It's almost like LLM output quality was never the point from a business perspective. Real people think in concepts and experiences instead of words. The words are not so important to get the idea across, but LLMs only model language. The problem is fundamental. There's no workaround. Averaging out word usage might even make the problem worse.
This sort of take is so tired and boring, and frankly has zero grounding in reality. "LLMs will never " is constantly being disproven every time they scale up to the next 10X and apply architectural improvements. Their internal representations are so cryptic and complex that even the top AI researchers don't really know how they work or what their limits are. No one is going to take you seriously as a rando HN user i…