Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Why does ChatGPT love the word "eager" so much?

news.ycombinator.com

Post reply on HN
Print view View original

51–60 of 69 posts

Re: Ask HN: Why does ChatGPT love the word "eager" so much?

#51

Post by bakuninsbart »

wietze wrote: post #7

Although "eager" isn't called out, a recent study of academic publications shows that the use of LLMs can be measured through word frequency analysis [1], finding certain words are disproportionally represented: > We study vocabulary changes in 14 million PubMed abstracts from 2010–2024, and show how the appearance of LLMs led to an abrupt increase in the frequency of certain style words. 1: https://arxiv.org/html/24…

The window of time where word frequency of chatgpt's favourites and usage of chatgpt is closely related is rather small I think. Academic language has a number of 'marker' words that are basically just style and will be more or less copied once you read many papers. 'Rigorous' is a general example, but most fields have their own. If many papers you read while writing your own paper use words like 'delve', you will be much more likely to use it yourself.

On another note, while the paper itself is pretty cool, in discussions on it I thought people where kind of looking down on using LLM's to help you write. There's a philistine moat in many fields around writing style. While writing well is in my experience correlated with paper quality, it is not predicated by it. And introducing tools that help people write more readable papers is probably a net benefit overall.

Re: Ask HN: Why does ChatGPT love the word "eager" so much?

#53

Post by solardev »

1attice wrote: post #28
JohnFen wrote: post #3

Earlier quoted context omitted.

> Eager isn't an especially uncommon word I'd say it's very common, at least in my part of the US. It's one of the words I hear on a daily basis, anyway. "Delve" used to be a very commonly used term before "deep dive" largely replaced it. I'm sure there are a whole lot of writings online that use "delve" because of the time period they were produced in. As a graybeard, I'm personally still much more likely to say "de…

As a former academic in tech I'm tickled that you think 'a deep dive' is normative while 'to delve' strikes your ear as strange. No judgment! I'm delighted, however, that language is so supple ("leverages domain-local synergies")

Probably a generational thing?

Re: Ask HN: Why does ChatGPT love the word "eager" so much?

#54

Post by mike_hearn »

cowsaymoo wrote: post #35
llm_nerd wrote: post #17

Earlier quoted context omitted.

There wasn't a study or analysis. It was just lazy speculation that felt good because it could be bound up in a "evil white countries exploiting the developing world" narrative. Where exploiting was "paying to do a job". It was submitted as https://news.ycombinator.com/item?id=40623629 Again, there is effectively zero real data showing this. Further, RLHF isn't likely to reinforce such word selection regardless. A mo…

tinfoil had theory: they implanted watermarks already, so that AI generated text can be flagged for future training runs or as a service, such that some phrases are coaxed to become statistical beacons.

That's not really a tinfoil hat theory. That's been possible for some years and OpenAI reportedly does watermark their outputs, and can detect it. They just haven't released it as a service because it'd annoy all the users who are using it for cheating :)

Re: Ask HN: Why does ChatGPT love the word "eager" so much?

#55

Post by 1attice »

solardev wrote: post #53
1attice wrote: post #28

Earlier quoted context omitted.

As a former academic in tech I'm tickled that you think 'a deep dive' is normative while 'to delve' strikes your ear as strange. No judgment! I'm delighted, however, that language is so supple ("leverages domain-local synergies")

Probably a generational thing?

OP cited youtube as 'delve'ers, which skews young, so I'm guessing it's that your cognitive 'ear' is tuned to the technosphere

Re: Ask HN: Why does ChatGPT love the word "eager" so much?

#56

Post by tommy_axle »

supercoffee wrote: post #20
solardev wrote: post #2

Can it just be a frequency illusion, where you tend to notice a new-to-you phenomenon again and again at first https://en.wikipedia.org/wiki/Frequency_illusion?wprov=sfla1 ? Eager isn't an especially uncommon word (eg "eager beavers" is a somewhat common saying), even though it's not used in most convos. I feel like "delve" is a YouTube phenomenon (as in "let's delve into this topic") as a weird proxy for "deep dive"…

My new frequency illusion is the phrase "ride or die". I'm 32 and I had never heard that in my life until I watched that show Cruel Summer and now it seems like I hear it everywhere, mostly on tv/ads/podcasts and almost never when talking to people I know. I refuse to say it because I don't fully understand what it means and when I do hear it, it always sounds a little cringey.

"Paramount" has been that word for me. Last year it seemed like every response from ChatGPT included it.

Re: Ask HN: Why does ChatGPT love the word "eager" so much?

#57

Post by famouswaffles »

jdietrich wrote: post #46
JohnMakin wrote: post #32

One of the funniest things I've ever heard/read about chatGPT "writing" is that it writes like a highschool student trying to inflate their word count on an essay. It typically uses a lot of words to say very little, and the style is hard to un-see once you recognize it.

ChatGPT will write in essentially any style you ask it to. The default behaviour is an artifact of RLHF training and the hidden pre-prompt, not a reflection of the capabilities of the underlying model. A pre-prompt along the following lines will radically change the nature of the output: "You are a subject matter expert in x. You communicate in a terse, information-dense manner. You make clear, strong arguments witho…

Yeah Open AI, Anthropic etc are all intentionally making LLMs sound more robotic than they are capable of sounding. It's pretty awful because the effects it has on fiction writing is disastrous.

Re: Ask HN: Why does ChatGPT love the word "eager" so much?

#58

Post by ben_w »

kgeist wrote: post #34
ben_w wrote: post #18

Earlier quoted context omitted.

Language models would be fairly useless for most people if they accurately modelled the source distribution, no better than autocomplete. In fact, they were fairly useless when they modelled the source distribution, that's why ChatGPT was an instant hit whereas GPT-3 was mainly only interesting to other AI reasearchers. What made LLMs suddenly interesting was that the responses were much more like answers and much le…

>In fact, they were fairly useless when they modelled the source distribution, that's why ChatGPT was an instant hit whereas GPT-3 was mainly only interesting to other AI reasearchers. I had a bot which used the original GPT3 (i.e. the completion model, not the chat model) and its answers were pretty decent (with the right prompting). Often even better than GPT3.5, whose answers were overly formulaic in comparison ("…

I think that means you would count as "another AI developer" ^_^;

Re: Ask HN: Why does ChatGPT love the word "eager" so much?

#59

Post by tivert »

llm_nerd wrote: post #17
PeterisP wrote: post #14

Earlier quoted context omitted.

I don't want to look for the source of analysis right now, but I recall reading a study demonstrating that a large part, if not most of the word frequency shift was caused by RLHF training done on data predominantly generated by people hired from lower income English-speaking countries which simply have a different dialect of English with a noticeably different frequency of certain phrases and expressions, so e.g. at…

There wasn't a study or analysis. It was just lazy speculation that felt good because it could be bound up in a "evil white countries exploiting the developing world" narrative. Where exploiting was "paying to do a job". It was submitted as https://news.ycombinator.com/item?id=40623629 Again, there is effectively zero real data showing this. Further, RLHF isn't likely to reinforce such word selection regardless. A mo…

> It was just lazy speculation that felt good because it could be bound up in a "evil white countries exploiting the developing world" narrative. Where exploiting was "paying to do a job".

Exploitation like that is in fact happening (see pretty much everything having to do with social media content moderation and RLHF to avoid disturbing content.

Also "paying to do a job" is not the moral panacea you seem to think it is.

Re: Ask HN: Why does ChatGPT love the word "eager" so much?

#60

Post by JojoFatsani »

JohnMakin wrote: post #32

One of the funniest things I've ever heard/read about chatGPT "writing" is that it writes like a highschool student trying to inflate their word count on an essay. It typically uses a lot of words to say very little, and the style is hard to un-see once you recognize it.

I have had good luck telling it to use simplified technical English.
Post reply on HN