I don't see that words often. Maybe it depends on the why question is asked.
Ask HN: Why does ChatGPT love the word "eager" so much?
31–40 of 69 posts
Re: Ask HN: Why does ChatGPT love the word "eager" so much?
#32One of the funniest things I've ever heard/read about chatGPT "writing" is that it writes like a highschool student trying to inflate their word count on an essay. It typically uses a lot of words to say very little, and the style is hard to un-see once you recognize it.
Re: Ask HN: Why does ChatGPT love the word "eager" so much?
#33The word ChatGPT uses the most is "apologize". Whenever I ask it to clean up a code and it either screws up the syntax or removes some necessary parts then starts apologizing until it is fixed or I give up. I specifically ask ChatGPT to stop apologizing because it becomes insufferable after a while.
For texts, it uses "furthermore" more than any other word followed by "lastly" imho.
Re: Ask HN: Why does ChatGPT love the word "eager" so much?
#34Earlier quoted context omitted.
I wonder why some words are overrepresented. Isn't the whole idea of language models to model word distribution as close as possible? Does it have something to do with RLHF? Or it's the training data?
Language models would be fairly useless for most people if they accurately modelled the source distribution, no better than autocomplete. In fact, they were fairly useless when they modelled the source distribution, that's why ChatGPT was an instant hit whereas GPT-3 was mainly only interesting to other AI reasearchers. What made LLMs suddenly interesting was that the responses were much more like answers and much le…
>In fact, they were fairly useless when they modelled the source distribution, that's why ChatGPT was an instant hit whereas GPT-3 was mainly only interesting to other AI reasearchers.
I had a bot which used the original GPT3 (i.e. the completion model, not the chat model) and its answers were pretty decent (with the right prompting). Often even better than GPT3.5, whose answers were overly formulaic in comparison ("as an AI language model...", "it's important to ..." all the time)
Re: Ask HN: Why does ChatGPT love the word "eager" so much?
#35Earlier quoted context omitted.
I don't want to look for the source of analysis right now, but I recall reading a study demonstrating that a large part, if not most of the word frequency shift was caused by RLHF training done on data predominantly generated by people hired from lower income English-speaking countries which simply have a different dialect of English with a noticeably different frequency of certain phrases and expressions, so e.g. at…
There wasn't a study or analysis. It was just lazy speculation that felt good because it could be bound up in a "evil white countries exploiting the developing world" narrative. Where exploiting was "paying to do a job". It was submitted as https://news.ycombinator.com/item?id=40623629 Again, there is effectively zero real data showing this. Further, RLHF isn't likely to reinforce such word selection regardless. A mo…
tinfoil had theory: they implanted watermarks already, so that AI generated text can be flagged for future training runs or as a service, such that some phrases are coaxed to become statistical beacons.
Re: Ask HN: Why does ChatGPT love the word "eager" so much?
#36Eager loading is a technical term with a specific meaning, contrasting with lazy loading. Be the text came out of an llm the real question for the user is, does this technical term actually to this situation. If it does, then it's an appropriate word choice carrying additional information.
It is generally the antonym, in technical contexts, to "lazy", as in "lazy evaluation" etc.
Re: Ask HN: Why does ChatGPT love the word "eager" so much?
#37Although "eager" isn't called out, a recent study of academic publications shows that the use of LLMs can be measured through word frequency analysis [1], finding certain words are disproportionally represented: > We study vocabulary changes in 14 million PubMed abstracts from 2010–2024, and show how the appearance of LLMs led to an abrupt increase in the frequency of certain style words. 1: https://arxiv.org/html/24…
To what extent can this style be overcome by prompting?
If it can be overcome in existing models, it’s probably going to involve different aspects including vocabulary, style, and organization.
Re: Ask HN: Why does ChatGPT love the word "eager" so much?
#38Can it just be a frequency illusion, where you tend to notice a new-to-you phenomenon again and again at first https://en.wikipedia.org/wiki/Frequency_illusion?wprov=sfla1 ? Eager isn't an especially uncommon word (eg "eager beavers" is a somewhat common saying), even though it's not used in most convos. I feel like "delve" is a YouTube phenomenon (as in "let's delve into this topic") as a weird proxy for "deep dive"…
So I just heard about Frequency Illusion earlier today, and now I see it here again. Such meta.
(Not kidding, from today’s NYT crossword column: https://www.nytimes.com/2024/08/11/crosswords/daily-puzzle-2...)
Re: Ask HN: Why does ChatGPT love the word "eager" so much?
#39What about "certainly" and "seasoned developer"?
Re: Ask HN: Why does ChatGPT love the word "eager" so much?
#40One of the funniest things I've ever heard/read about chatGPT "writing" is that it writes like a highschool student trying to inflate their word count on an essay. It typically uses a lot of words to say very little, and the style is hard to un-see once you recognize it.
It's called thinking out loud. Gets you better results. Try it out yourself.