Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why wordfreq will not be updated

github.com

Post reply on HN
Print view View original

491–500 of 542 posts

Re: Why wordfreq will not be updated

#491

Post by appendix-rock »

cdrini wrote: post #481

This has to be the most annoying hacker news comment section I've ever seen. It's just the same ~4 viewpoints rehashed again, and again, and again. Why don't folks just upvote other comments that say the same thing instead of repeating the same things? And now a hopefully new comment: having a word frequency measure of the internet as we're going into AI being more used would be IMMENSELY useful specifically _because…

They want to display how they’re truly intelligent (unlike LLMs) by checks notes rehashing opinions that they’ve read millions of times online.

Sound familiar to anyone?

Re: Why wordfreq will not be updated

#492

Post by QRe »

brunokim wrote: post #353
QRe wrote: post #351

I understand the frustration shared in this post but I wholeheartedly disagree with the overall sentiment that comes with it. The web isn't dead, (Gen)AI, SEO, spam and pollution didn't kill anything. The world is chaotic and net entropy (degree of disorder) of any isolated or closed system will always increase. Same goes for the web. We just have to embrace it and overcome the challenges that come with it.

Here is an expert saying there is a problem and how it killed its research effort, and yet you say that things are the same as ever and nothing was killed.

1. I am not discrediting the expert in any way, if anything, I think their decision to quit is understandable - there is now a challenge that arose during his research that is not in their interest to pursue (information pollution is not research in corpus linguistics / NLP).

2. I never said that things are the same as ever, quite the opposite actually. I am saying the world evolves constantly. It's naive to say company X/Y/Z killed something or made something unusable, when there is constant inevitable change. We should focus on how to move forward giving this constraint, and not dwell on times where the web was so much 'cleaner' and 'nicer', more manageable etc.

Re: Why wordfreq will not be updated

#493

Post by bakugo »

devjab wrote: post #479
avazhi wrote: post #466

I agree with the general ethos of the piece (albeit a few of the details are puzzling and unnecessarily partisan - content on X isn't invariably worthless drivel, nor does what Reddit is doing make much intellectual as opposed to economic [IPO-influenced] sense - but this line: 'OpenAI and Google can collect their own damn data. I hope they have to pay a very high price for it, and I hope they're constantly cursing t…

> puzzling and unnecessarily partisan - content on X isn't invariably worthless drivel Maybe this is because I’m European, but what is partisan about calling X invariably worthless drivel? Seems a lot like facts to me considering what has been going on with the platform moderation since Elon Musk bought it. It’s so bad that the EU consider it a platform for misinformation these days.

> It’s so bad that the EU consider it a platform for misinformation these days.

Can you define "misinformation"? Is it just things the government disagrees with?

Re: Why wordfreq will not be updated

#494

Post by advael »

cdrini wrote: post #481

This has to be the most annoying hacker news comment section I've ever seen. It's just the same ~4 viewpoints rehashed again, and again, and again. Why don't folks just upvote other comments that say the same thing instead of repeating the same things? And now a hopefully new comment: having a word frequency measure of the internet as we're going into AI being more used would be IMMENSELY useful specifically _because…

On a meta level I agree that having this kind of dataset with "before and after" would be pretty interesting. On an object level I do not predict that this would increase the overall diversity of language usage - and in fact it would be extremely surprising if this was even possible due to some general mathematical properties of neural networks - nor would "more professional writing," though I do agree with this characterization of the way AI-generated text sounds. The more I work with LLMs and encounter them in the wild, the greater my confidence that I can tell when something was generated, with the exception of B2B marketing copy and communications from HR departments or state agencies

On the level of meta-discourse you seem to want to also speak to: Dang even when people have the Official Corporate Approved Perspective (in particular, the claim that it's "like being able to chat with someone that has an infinite vocabulary" is probably the silliest delusional AI hype I've heard all week) and the most upvotes in the thread they still think they're an embattled ideological minority. Starting to think that literally zero people in the modern world don't have or affect a victim complex of some kind

Re: Why wordfreq will not be updated

#495

Post by wbillingsley »

appendix-rock wrote: post #491
cdrini wrote: post #481

This has to be the most annoying hacker news comment section I've ever seen. It's just the same ~4 viewpoints rehashed again, and again, and again. Why don't folks just upvote other comments that say the same thing instead of repeating the same things? And now a hopefully new comment: having a word frequency measure of the internet as we're going into AI being more used would be IMMENSELY useful specifically _because…

They want to display how they’re truly intelligent (unlike LLMs) by checks notes rehashing opinions that they’ve read millions of times online. Sound familiar to anyone?

I wonder whether future generations will be ingrained with a Truman Show fear that maybe only the few thousand people they meet are real and everything else is generated background noise.

Re: Why wordfreq will not be updated

#496

Post by avazhi »

devjab wrote: post #479
avazhi wrote: post #466

I agree with the general ethos of the piece (albeit a few of the details are puzzling and unnecessarily partisan - content on X isn't invariably worthless drivel, nor does what Reddit is doing make much intellectual as opposed to economic [IPO-influenced] sense - but this line: 'OpenAI and Google can collect their own damn data. I hope they have to pay a very high price for it, and I hope they're constantly cursing t…

> puzzling and unnecessarily partisan - content on X isn't invariably worthless drivel Maybe this is because I’m European, but what is partisan about calling X invariably worthless drivel? Seems a lot like facts to me considering what has been going on with the platform moderation since Elon Musk bought it. It’s so bad that the EU consider it a platform for misinformation these days.

Because the author specifically mentioned that it's worthless because it's 'right-wing' (a 'right-wing cesspool'), as if there aren't plenty of people espousing left-wing views on the platform. The right-wing comment in particular is what makes the statement blatantly partisan.

Re: Why wordfreq will not be updated

#497

Post by BiteCode_dev »

cdrini wrote: post #481

This has to be the most annoying hacker news comment section I've ever seen. It's just the same ~4 viewpoints rehashed again, and again, and again. Why don't folks just upvote other comments that say the same thing instead of repeating the same things? And now a hopefully new comment: having a word frequency measure of the internet as we're going into AI being more used would be IMMENSELY useful specifically _because…

Also it breaks the languagr barreer, you can now read the Chinese internet if you want, or chat transparently in Arabic. That's going to be interesting.

Re: Why wordfreq will not be updated

#498

Post by pbhjpbhj »

bongodongobob wrote: post #352
ziml77 wrote: post #349

Earlier quoted context omitted.

I've reached a point where even if my first reaction to a photo is to be impressed, I then quickly think "oh but what it this is AI?" and then immediately my excitement for the photo is ruined because it may not actually be a photo at all.

I don't get that perspective at all. Who cares what made it.

You don't find a difference between things that exist and things that don't?

Re: Why wordfreq will not be updated

#499

Post by cdrini »

vlan121 wrote: post #490
cdrini wrote: post #481

This has to be the most annoying hacker news comment section I've ever seen. It's just the same ~4 viewpoints rehashed again, and again, and again. Why don't folks just upvote other comments that say the same thing instead of repeating the same things? And now a hopefully new comment: having a word frequency measure of the internet as we're going into AI being more used would be IMMENSELY useful specifically _because…

You haven't read the whole thing. It says that: or that could benefit generative AI.

I did read it :) not sure how that line applies here, can you expand?
Post reply on HN