Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why wordfreq will not be updated

github.com

Post reply on HN
Print view View original

231–240 of 542 posts

Re: Why wordfreq will not be updated

#231

Post by bongodongobob »

squigz wrote: post #217
solardev wrote: post #211

Earlier quoted context omitted.

Have "good" small webs EVER prevailed? Every content system seems to get polluted by noise once it hits mainstream usage: IRC, Usenet, reddit, Facebook, geocities, Yahoo, webrings, etc. Once-small curated selections eventually grow big enough to become victims of their own successes and taken over by spam. It's always an arms race of quality vs quantity, and eventually the curators can't keep up with the sheer volume…

> Have "good" small webs EVER prevailed? You ask on HN, one of the highest quality sites I've ever visited in any age of the Internet. IRC is still alive and well among pretty much the same audience as always. I'm not sure it's fair to compare that with the others.

It's high quality when the content is within HN's bubble. Anything related to health, politics, or Microsoft is full of misinformation, ignorance, and garbage like any other site. The Microsoft discussions in particular are extremely low quality.

Re: Why wordfreq will not be updated

#232

Post by halostatue »

grues-dinner wrote: post #130
shagie wrote: post #83

Earlier quoted context omitted.

I wish more people presented recipes like cooking for engineers. For example - Meat Lasagna https://www.cookingforengineers.com/recipe/36/Meat-Lasagna

And here I thought my defacement of printed recipes by bracketing everything that goes together at each stage was just me. There are, well, maybe not dozens but at least two of us! Saves a lot of bowls when you know without further checking that you can, say, just dump the flour and sugar, butter and eggs into the big bowl without having to prepare separately because they're in the "1: big bowl" bracket.

Depends on what you’re doing. For best cookies, you want to cream the butter with the sugar, then add the eggs, and finally add the flour. If you’re interested and can find one, it’s worth taking a vegan baking class. You learn a lot about ingredient substitutions for baking, about what the different non-vegan ingredients are doing that you have to compensate for…and it does something that I’ve only recently started seeing happen in non-vegan baking recipes: it separates the wet ingredients from the dry ingredients.

That is, when baking, you can usually (again, exceptions for creaming the sugar in butter, etc.) take all of your dry ingredients and mix/sift them together, and then you pour your wet ingredients in a well you’ve made in the dry ingredients (these can also usually be mixed together).

Re: Why wordfreq will not be updated

#234

Post by squigz »

bongodongobob wrote: post #231
squigz wrote: post #217

Earlier quoted context omitted.

> Have "good" small webs EVER prevailed? You ask on HN, one of the highest quality sites I've ever visited in any age of the Internet. IRC is still alive and well among pretty much the same audience as always. I'm not sure it's fair to compare that with the others.

It's high quality when the content is within HN's bubble. Anything related to health, politics, or Microsoft is full of misinformation, ignorance, and garbage like any other site. The Microsoft discussions in particular are extremely low quality.

I disagree. Even politics spurs intelligent, nuanced discussion here on HN.

And to hold up discussions about MS as an example of 'extremely' low quality discussion is, ah, interesting. Do you have any recent examples of such discussions?

Re: Why wordfreq will not be updated

#235

Post by bee_rider »

jerf wrote: post #124
olabyne wrote: post #85

Earlier quoted context omitted.

I never thought about that. Humans losing their ability to detect AI content from reality ? It's frightening.

It's even worse than that. Most people have no idea how far CGI has come, and how easily it is wielded even by a couple of dedicated teens on their home computer, let alone people with a vested interest in faking something for some financial reason. People think they know what a "special effect" looks like, and for the most part, people are wrong . They know what CGI being used to create something obviously impossibl…

Plus, movies continue (for some reason) to be made with very bad and obvious CGI, leading people to believe all CGI is easy to spot.

Re: Why wordfreq will not be updated

#236

Post by rovr138 »

solardev wrote: post #222
squigz wrote: post #217

Earlier quoted context omitted.

> Have "good" small webs EVER prevailed? You ask on HN, one of the highest quality sites I've ever visited in any age of the Internet. IRC is still alive and well among pretty much the same audience as always. I'm not sure it's fair to compare that with the others.

Well, niche forums are kinda different when they manage to stay small and niche. Not just HN but car forums, LED forums, etc. But if they ever include other topics, they risk becoming more mainstream and noisy. Even within adjacent fields (like the various Stacks) it gets pretty bad. Maybe the trick is to stay within a single small sphere then and not become a general purpose discussion site? And to have a low enough…

Yes. That's the small web.

A good example of the generalization problem you discuss is reddit.

You have to unsubscribe from all the defaults and find the small, niche, communities about specific topics. If not, it's the same stuff, reposted, over and over, across different subs and/or social sites.

Re: Why wordfreq will not be updated

#237

Post by squigz »

solardev wrote: post #222
squigz wrote: post #217

Earlier quoted context omitted.

> Have "good" small webs EVER prevailed? You ask on HN, one of the highest quality sites I've ever visited in any age of the Internet. IRC is still alive and well among pretty much the same audience as always. I'm not sure it's fair to compare that with the others.

Well, niche forums are kinda different when they manage to stay small and niche. Not just HN but car forums, LED forums, etc. But if they ever include other topics, they risk becoming more mainstream and noisy. Even within adjacent fields (like the various Stacks) it gets pretty bad. Maybe the trick is to stay within a single small sphere then and not become a general purpose discussion site? And to have a low enough…

I'm not entirely sure it's about content (while HN is certainly tech-focused, politics, health, philosophy all come up with regularity) or even content moderation, although they both certainly play a part (particularly the moderation around here. Thanks, staff!)

I wonder if it is more to do with the community itself. HN users tend to have very intelligent discussions on pretty much anything, and discourages shitty, unnuanced, one-line takes. This, coupled with a healthy moderation system, makes it hard for the lower quality discussion to break in and override the good stuff.

Re: Why wordfreq will not be updated

#238

Post by 38 »

solardev wrote: post #211
jll29 wrote: post #203

I regret the situation led to the OP feel discourage about the NLP community, wo which I belong, and I just want to say "we're not all like that", even though it is a trend and we're close to peak hype (slightly past even?). The complaint about pollution of the Web with artificial content is timely, and it's not even the first time due to spam farms intended to game PageRank, among other nonsense. This may just mean…

Have "good" small webs EVER prevailed? Every content system seems to get polluted by noise once it hits mainstream usage: IRC, Usenet, reddit, Facebook, geocities, Yahoo, webrings, etc. Once-small curated selections eventually grow big enough to become victims of their own successes and taken over by spam. It's always an arms race of quality vs quantity, and eventually the curators can't keep up with the sheer volume…

its so easy to solve this problem, not sure why anyone hasnt done it yet.

1. build a userbase, free product

2. once userbase get big enough, any new account requires a monthly fee, maybe $1

3. keep raising the fee higher and higher, until you get to the point that the userbase is manageable.

no ads, simple.

Re: Why wordfreq will not be updated

#239

Post by bongodongobob »

diggan wrote: post #225

One of the examples is the increased usage of "delve" which Google Trends confirms increased in usage since 2022 (initial ChatGPT release): https://trends.google.com/trends/explore?date=all&q=delve&hl... It seems however it started increasing most in usage just these last few months, maybe people are talking more about "delve" specifically because of the increase in usage? A usage recursion of some sorts.

Delves are a new thing in World of Warcraft released 9/10 this year. Delve is also an M365 product that has been around for some time and is being discontinued in December. So no, that has nothing to do with LLMs.

Re: Why wordfreq will not be updated

#240

Post by Suppafly »

jsheard wrote: post #80
kevindamm wrote: post #32

Earlier quoted context omitted.

Yes but not quite as far as you imply. The training data is weighted by a quality metric, articles written by journalists and wikipedia contributors are given more weight than Aunt May's brownie recipe and corpoblogspam.

> The training data is weighted by a quality metric At least in Googles case, they're having so much difficulty keeping AI slop out of their search results that I don't have much faith in their ability to give it an appropriately low training weight. They're not even filtering the comically low-hanging fruit like those YouTube channels which post a new "product review" every 10 minutes, with an AI generated thumbnail…

>At least in Googles case, they're having so much difficulty keeping AI slop out of their search results that I don't have much faith in their ability to give it an appropriately low training weight.

I've noticed that lately. It used to be the top google result was almost always what you needed. Now at the top is an AI summary that is pretty consistently wrong, often in ways that aren't immediately obvious if you aren't familiar with the topic.

Post reply on HN