Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why wordfreq will not be updated

github.com

Post reply on HN
Print view View original

251–260 of 542 posts

Re: Why wordfreq will not be updated

#251

Post by bongodongobob »

squigz wrote: post #248
bongodongobob wrote: post #245

Earlier quoted context omitted.

I hide every single article about MS because it's filled with all the neckbeardy tropes about their products being garbage spyware, switch to Linux, they're stealing your data, the OS is trash etc. It's comments from people who have never managed large scale MS based environments comparing their Windows Home to the other 90% of the business ecosystem that has nothing to do with home users or MS's main cash cow, busin…

This is a funny comment in a thread about low quality discussion.

I'm describing why I no longer engage with MS related posts.

Re: Why wordfreq will not be updated

#252

Post by sergiotapia »

umvi wrote: post #210
ziddoap wrote: post #153

Earlier quoted context omitted.

Steel without nuclear contamination is sought after, and only available from pre-war / pre-atomic sources. The analogy is that data is now contaminated with AI like steel is now contaminated with nuclear fallout. https://en.wikipedia.org/wiki/Low-background_steel > Low-background steel, also known as pre-war steel[1] and pre-atomic steel,[2] is any steel produced prior to the detonation of the first nuclear bombs in…

> and only available from pre-war / pre-atomic sources. From the same wiki you linked: "Since the end of atmospheric nuclear testing, background radiation has decreased to very near natural levels, making special low-background steel no longer necessary for most radiation-sensitive uses, as brand-new steel now has a low enough radioactive signature" and "For the most demanding items even low-background steel can be t…

reading stuff like this makes me so happy. no matter how fucked up something may be there is always a way to clean right up.

Re: Why wordfreq will not be updated

#253

Post by jhbadger »

jerf wrote: post #124
olabyne wrote: post #85

Earlier quoted context omitted.

I never thought about that. Humans losing their ability to detect AI content from reality ? It's frightening.

It's even worse than that. Most people have no idea how far CGI has come, and how easily it is wielded even by a couple of dedicated teens on their home computer, let alone people with a vested interest in faking something for some financial reason. People think they know what a "special effect" looks like, and for the most part, people are wrong . They know what CGI being used to create something obviously impossibl…

And you see things like the The Lion King remake or its upcoming prequel being called "live action" because it doesn't look like a cartoon like the original. But they didn't film actual lions running around -- it's all CGI.

Re: Why wordfreq will not be updated

#254

Post by [deleted] »

38 wrote: post #238
solardev wrote: post #211

Earlier quoted context omitted.

Have "good" small webs EVER prevailed? Every content system seems to get polluted by noise once it hits mainstream usage: IRC, Usenet, reddit, Facebook, geocities, Yahoo, webrings, etc. Once-small curated selections eventually grow big enough to become victims of their own successes and taken over by spam. It's always an arms race of quality vs quantity, and eventually the curators can't keep up with the sheer volume…

its so easy to solve this problem, not sure why anyone hasnt done it yet. 1. build a userbase, free product 2. once userbase get big enough, any new account requires a monthly fee, maybe $1 3. keep raising the fee higher and higher, until you get to the point that the userbase is manageable. no ads, simple.

[deleted]

Re: Why wordfreq will not be updated

#255

Post by zaik »

If generative AI has a significantly different word frequency from humans then it also shouldn't be hard to detect text written generative AI. However my last information is that tools to detect text written by generative AI are not that great.

Re: Why wordfreq will not be updated

#256

Post by BeFlatXIII »

ToucanLoucan wrote: post #66
voytec wrote: post #13

I agree in general but the web was already polluted by Google's unwritten SEO rules. Single-sentence paragraphs, multiple keyword repetitions and focus on "indexability" instead of readability, made the web a less than ideal source for such analysis long before LLMs. It also made the web a less than ideal source for training. And yet LLMs were still fed articles written for Googlebot, not humans. ML/LLM is the second…

This feels like a second, magnitudes larger Eternal September. I wonder how much more of this the Internet can take before everyone just abandons it entirely. My usage is notably lower than it was in even 2018, it's so goddamn hard to find anything worth reading anymore (which is why I spend so much damn time here, tbh).

I hope this trend accelerates to force us all into grass-touching and book-reading. The sooner, the better.

Re: Why wordfreq will not be updated

#257

Post by ljsprague »

Ensorceled wrote: post #185
Diti wrote: post #122

Earlier quoted context omitted.

As far as I noticed, the “right” effectively gets the boot in most online communities which abide to a Code of Conduct, leaving mostly the “left” (the most recent example I have in mind of such moderation efforts is the save-nix-together.org open letter). It’s interesting that you don’t notice this happening in the communities you seem to frequent.

No, I see a lot of the "right" discourse. Many are openly supporting Putin now. I follow many conservative (US) pundits and journalists and they have either taken a hard right turn or are raising the alarm and supporting Harris. I see similar trends here in Canada. Yes, I see that the left has become more authoritarian, but it pales to the hard shift I see on the right.

Something inherently wrong with supporting Putin?

Re: Why wordfreq will not be updated

#258

Post by felbane »

sergiotapia wrote: post #252
umvi wrote: post #210

Earlier quoted context omitted.

> and only available from pre-war / pre-atomic sources. From the same wiki you linked: "Since the end of atmospheric nuclear testing, background radiation has decreased to very near natural levels, making special low-background steel no longer necessary for most radiation-sensitive uses, as brand-new steel now has a low enough radioactive signature" and "For the most demanding items even low-background steel can be t…

reading stuff like this makes me so happy. no matter how fucked up something may be there is always a way to clean right up.

glances nervously at atmospheric CO2

Re: Why wordfreq will not be updated

#259

Post by BeFlatXIII »

olabyne wrote: post #85
weinzierl wrote: post #64

"I don't think anyone has reliable information about post-2021 language usage by humans." We've been past the tipping point when it comes to text for some time, but for video I feel we are living through the watershed moment right now. Especially smaller children don't have a good intuition on what is real and what is not. When I get asked if the person in a video is real, I still feel pretty confident to answer but…

I never thought about that. Humans losing their ability to detect AI content from reality ? It's frightening.

It's a defense lawyer's dream.

Re: Why wordfreq will not be updated

#260

Post by andai »

Has anyone taken a look at a random sample of web data? It's mostly crap. I was thinking of making my own search engine, knowledge database etc based on a random sample of web pages, but I found that almost all of them were drivel. Flame wars, asinine blog posts, and most of all, advertising. Forget spam, most of the legit pages are trying to sell something too!

The conclusion I arrived at was that making my own crawler actually is feasible (and given my goals, necessary!) because I'm only interested in a very, very small fraction of what's out there.

Post reply on HN