[flagged]
> Also, it is shocking how authoritarian the “left” has become in my lifetime.
We are going through a general uptick in authoritarian "discussions" online. It's interesting that you are only seeing it on the "left".
Why wordfreq will not be updated
91–100 of 542 posts
Re: Why wordfreq will not be updated
#92Enshittification is accelerating. A good 70% of my Facebook feed is now obviously AI generated images with AI generated text blurbs that have nothing to do with the accompanying images likely posted by overseas bot farms. I'm also noticing more and more "books" on Amazon that are clearly AI generated and self published.
It's okay. Amazon has limited authors to self publishing only 3 books per day (yes, really). That will surely solve the problem.
I read that as 3 books per year at first and thought to myself that that was a rather harsh limitation but surely any true respectable author wouldn't be spitting more than that...
...and then I realized you wrote 3 books a day. What the hell.
Re: Why wordfreq will not be updated
#93Re: Why wordfreq will not be updated
#94I really like the fact that the content of the conventional user content internet is becoming willfully polluted and ever more useless by the incessant influx of "ai"-garbage. At some point all of this will become so awful that nerds will create new and quiet corners of real people and real information while the idiot rabble has to use new and expensive tools peddled by scammy tech bros to handle the stench of automated manure that flows out of stagnant llms digesting themselves.
Re: Why wordfreq will not be updated
#95I really like the fact that the content of the conventional user content internet is becoming willfully polluted and ever more useless by the incessant influx of "ai"-garbage. At some point all of this will become so awful that nerds will create new and quiet corners of real people and real information while the idiot rabble has to use new and expensive tools peddled by scammy tech bros to handle the stench of automa…
Most of the time, HN is that quiet corner. I just hope it stays that way.
Re: Why wordfreq will not be updated
#96Earlier quoted context omitted.
Yes but not quite as far as you imply. The training data is weighted by a quality metric, articles written by journalists and wikipedia contributors are given more weight than Aunt May's brownie recipe and corpoblogspam.
It certainly feels like the amount of regurgitated, nonsensical, generated content (nontent?) has risen spectacularly specifically in the past few years. 2021 sounds about right based on just my own experience, even though I can't point to any objective source backing that up.
Ooh I like “nontent.” Nothing like a spicy portmanteau!
Re: Why wordfreq will not be updated
#97Earlier quoted context omitted.
I guess it would be interesting but differentiating pollution from language evolution seems very tricky since getting a non polluted corpus gets harder and harder
Arguably it is a form of language evolution. I bet humans have started using "delve" more too, on average. I think the best we can do is look at the trends and think about potential causes.
> I bet humans have started using "delve" more too, on average.
I wish there were a way to check.
Re: Why wordfreq will not be updated
#98Earlier quoted context omitted.
There seem to be quite a few recipe sharing sites around - e.g. allrecipes.com.
And they're all flooded with low effort trash and useless. The only remaining reliable source - now that many newspapers are axing the remaining staff in favour of LLMs - is pre-2020 print cookbooks. Anything online or printed later must be assumed to be tainted, full of untested sewage and potentially dangerous suggestions.
The wife and I use the internet for recipe ideas... but we hardly ever follow them directly anymore. We're no formally-trained chefs but we've been home cooks for over 20 years now, and so many of them are self-evidently bad, or distinctly suboptimal. The internet chef's aversion to flavor is a meme with us now; "add one-sixty-fourth of a teaspoon of garlic powder to your gallon of soup, and mix in two crystals of table salt". Either that or they're all getting some seriously potent spices all the time and I'd like to know where they shop because my spices are nowhere near as powerful as theirs.
Re: Why wordfreq will not be updated
#99I created https://lowbackgroundsteel.ai/ in 2023 as a place to gather references to unpolluted datasets. I'll add wordfreq. Please submit stuff to the Tumblr.
[deleted]
Re: Why wordfreq will not be updated
#100Intuitively I feel like word frequency would be one of the things least impacted by LLM output, no?