Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why wordfreq will not be updated

github.com

Post reply on HN
Print view View original

211–220 of 542 posts

Re: Why wordfreq will not be updated

#211

Post by solardev »

jll29 wrote: post #203

I regret the situation led to the OP feel discourage about the NLP community, wo which I belong, and I just want to say "we're not all like that", even though it is a trend and we're close to peak hype (slightly past even?). The complaint about pollution of the Web with artificial content is timely, and it's not even the first time due to spam farms intended to game PageRank, among other nonsense. This may just mean…

Have "good" small webs EVER prevailed?

Every content system seems to get polluted by noise once it hits mainstream usage: IRC, Usenet, reddit, Facebook, geocities, Yahoo, webrings, etc. Once-small curated selections eventually grow big enough to become victims of their own successes and taken over by spam.

It's always an arms race of quality vs quantity, and eventually the curators can't keep up with the sheer volume anymore.

Re: Why wordfreq will not be updated

#212

Post by left-struck »

ckemere wrote: post #84
baq wrote: post #29

Earlier quoted context omitted.

Transformers read a lot during training, it might actually be beneficial for the companies to the point those works never see the light of day, only machines would read them. That's so dystopian I'd say those works should be published so they eventually get into the public domain.

Rooms full of people writing into a computer is a striking mental picture. It feels like it could be background for a great plot for a book/movie.

Have you heard of Severance? This has a vibe extremely similar to that show.

Re: Why wordfreq will not be updated

#213

Post by exo-pla-net »

antirez wrote: post #171

Ok so post author is AI skeptic and this is his retaliation, likely because his work is affected. I believe governments should address the problem with welfare but being against technical advances is always being in the wrong side of history.

This is a tech site, where >50% of us are programmers who have achieved greater productivity thanks to LLM advances.

And yet we're filled to the gills with Luddite sentiments and AI content fearmongering.

Imagine the hysteria and the skull-vibrating noise of the non-HN rabble when they come to understand where all of this is going. They're going to do their darndest to stop us from achieving post-economy.

Re: Why wordfreq will not be updated

#214

Post by left-struck »

aucisson_masque wrote: post #18

Did we (the humans) somehow managed to pollute the internet so much with AI that's it's now barely usable ? In my opinion the internet can be considered as the equivalent of a natural environment like the earth. it's a space where people share, meet, talk, etc. I find it astonishing that after polluting our natural environment we know polluted the internet.

>We the humans

Nice try

If it’s not clear, I’m joking.

Re: Why wordfreq will not be updated

#216

Post by left-struck »

Lvl999Noob wrote: post #36
aucisson_masque wrote: post #24

It could be used to spot LLM generated text. compare the frequency of words to those used in human natural writings and you spot the computer from the human.

It could be used to differentiate LLM text from pre-LLM human text maybe. The thing, our AIs may not be very good at learning but our brains are. The more we use AI, the more we integrate LLMs and other tools into our life, the more their output will influence us. I believe there was a study (or a few anecdotes) where college papers checked for AI material were marked AI written even though they were written by human…

> The more we use AI, the more we integrate LLMs and other tools into our life, the more their output will influence us

Hmm I don’t disagree but I think it will be valuable skill going forward to write text that doesn’t read like it was written by an LLM

This is an arms race that I’m not sure we can win though. It’s almost like a GAN.

Re: Why wordfreq will not be updated

#217

Post by squigz »

solardev wrote: post #211
jll29 wrote: post #203

I regret the situation led to the OP feel discourage about the NLP community, wo which I belong, and I just want to say "we're not all like that", even though it is a trend and we're close to peak hype (slightly past even?). The complaint about pollution of the Web with artificial content is timely, and it's not even the first time due to spam farms intended to game PageRank, among other nonsense. This may just mean…

Have "good" small webs EVER prevailed? Every content system seems to get polluted by noise once it hits mainstream usage: IRC, Usenet, reddit, Facebook, geocities, Yahoo, webrings, etc. Once-small curated selections eventually grow big enough to become victims of their own successes and taken over by spam. It's always an arms race of quality vs quantity, and eventually the curators can't keep up with the sheer volume…

> Have "good" small webs EVER prevailed?

You ask on HN, one of the highest quality sites I've ever visited in any age of the Internet.

IRC is still alive and well among pretty much the same audience as always. I'm not sure it's fair to compare that with the others.

Re: Why wordfreq will not be updated

#218

Post by squigz »

jll29 wrote: post #203

I regret the situation led to the OP feel discourage about the NLP community, wo which I belong, and I just want to say "we're not all like that", even though it is a trend and we're close to peak hype (slightly past even?). The complaint about pollution of the Web with artificial content is timely, and it's not even the first time due to spam farms intended to game PageRank, among other nonsense. This may just mean…

> people consuming auto-generated content to keep the masses from away from critical thinking. This is now happening

The people who stay away from critical thinking were doing that already and will continue to do so, 'AI' content or not.

Re: Why wordfreq will not be updated

#219

Post by FrustratedMonky »

commodoreboxer wrote: post #197
robrtsql wrote: post #167

Earlier quoted context omitted.

I assume FrustratedMonky's comment was satirical, given that it appears to have been written like an LLM and starts with a "but, but, but" which is how you might represent someone you disagree with presenting their argument.

I thought so, but the rest of the comment is worded quite reasonably, so I decided to not interpret it as hyperbole or irony.

"quite reasonably"

Joke is, it was GPT 4o. I didn't phrase it to be like an LLM, I asked GPT to defend itself, and used it's text un-edited. Except adding the 'but but but"

There is a lot out there that people are no longer able to determine if it is an LLM. They are getting better. That they are no longer obvious is the scary part.

Re: Why wordfreq will not be updated

#220

Post by voytec »

umvi wrote: post #208
voytec wrote: post #184

Earlier quoted context omitted.

To whomever downvoted parent: please don't act against people brave enough to state that they don't know something. This is a desired quality, increasingly less present in IT work environments. People afraid of being shamed for stating knowledge gaps are not the folks you want to work with.

I feel like there's a minimum "due diligence" bar to meet though before asking, otherwise it comes across as "I'm too lazy to google the reference and connect the dots myself, but can someone just go ahead and distill a nice summary for me"

In this particular case, I was out of the loop regarding the clever analogy myself. I'm now a tad smarter because someone else expressed lack of understanding, and I learned from responses to this (grayed due to downvotes) comment.
Post reply on HN