"Sure, there was spam in the wordfreq data sources, but it was manageable and often identifiable."
How sure can we be about that?
Why wordfreq will not be updated
341–350 of 542 posts
Re: Why wordfreq will not be updated
#342I'm going to call it: The Web is dead. Thanks to "AI" I spend more time now digging through searches trying to find something useful than I did back in 2005. And the sites you do find are largely garbage. As a random example: just trying to find a particular popular set of wireless earbuds takes me at least 10 minutes, when I already know the company, the company's website, other vendors that sell the company's goods…
The web is much more than a shopping site.
It is, but the SEO spammers who ruined the web want it to be shopping mall, and they can't even do a particularly good job at being one.
Re: Why wordfreq will not be updated
#343Earlier quoted context omitted.
IMO HN actually scores quite highly in terms of health/politics and so forth content because the both mainstream and fringe ideas get both shown and pushback. A vaping discussion brought up glycerin used was safe and the same thing used in smoke machines and someone else brought up a study showing that smoke machines are an occasional safety issue. Nowhere near every discussion goes that well but stick around and you…
As a software engineer married to a healthcare professional, I disagree strongly about the quality of the healthcare discussions here. A whole lot of the conversation is software engineers who think that they can reason from first principles in two minutes about this thing that professionals dedicate their whole lives to mastering, and who therefore don't understand the most basic concepts of the field. Sometimes I t…
> about this thing that professionals dedicate their whole lives to mastering
After doing some healthcare work I ended up understanding that some topics are not well known even by the professionals dedicating their whole lives to that because there are big gaps in the human knowledge on the topics.
I agree that people that think they can reason in two minutes about anything are a problem, but it's not a healthcare only issue (same happens for politics, economics, environment, etc.)
Engineers have the luck to work in the field where many things have a clear, known explanation (although, try to make an estimation about how long a team will implement a feature, and everybody will come up with something else).
Re: Why wordfreq will not be updated
#344Earlier quoted context omitted.
This is a common survivorship bias fallacy since you only notice the bad CGI. I'm certain you'd be shocked to see the amount of CG that's in some of your favorite movies made in the last ~10-20 years that you didn't notice because it's undetectable
I won’t be, I’m aware that lots of movies are mostly CGI. But, yeah, I do think it is some kind of bias. Maybe not survivorship, though… maybe it is a generalized sort of Malmquist bias? Like the measurement is not skewed by the tendency of movies with good CGI to go away. It is skewed by the fact that bad CGI sticks out.
Actually wait I take it back, I mean, I was aware that lots of Digital Touch-up happens in movie sets, more than lots of people might expect, and more often that one might expect even in mundane movies, but even still, this comment’s video was pretty shocking anyway.
Re: Why wordfreq will not be updated
#345Earlier quoted context omitted.
This is a common survivorship bias fallacy since you only notice the bad CGI. I'm certain you'd be shocked to see the amount of CG that's in some of your favorite movies made in the last ~10-20 years that you didn't notice because it's undetectable
This is an amazing demo reel of effects shots used in "mundane" TV shows - comedies and produce procedurals. - for faking locations. https://www.youtube.com/watch?v=clnozSXyF4k
That is really something even as somebody who expects lots of CGI touch-up in sets.
Re: Why wordfreq will not be updated
#346I agree in general but the web was already polluted by Google's unwritten SEO rules. Single-sentence paragraphs, multiple keyword repetitions and focus on "indexability" instead of readability, made the web a less than ideal source for such analysis long before LLMs. It also made the web a less than ideal source for training. And yet LLMs were still fed articles written for Googlebot, not humans. ML/LLM is the second…
This feels like a second, magnitudes larger Eternal September. I wonder how much more of this the Internet can take before everyone just abandons it entirely. My usage is notably lower than it was in even 2018, it's so goddamn hard to find anything worth reading anymore (which is why I spend so much damn time here, tbh).
[deleted]
Re: Why wordfreq will not be updated
#347Sad. I'd love to see by how much the use of world "delve" has increased since 2021...
Google ngram viewer, perhaps?
Re: Why wordfreq will not be updated
#348Earlier quoted context omitted.
They likely never started critically thinking, so they never had to get started on not doing so. (If children are never taught to think critically, then...)
It's almost like its a systemic failure that is artificially created so that people wont think critically... hmmm
> is artificially created
You imply that thousands of year ago everybody was thinking critically?
Thinking critically is hard, stressful and might take some joy from your life.
Re: Why wordfreq will not be updated
#349Earlier quoted context omitted.
I never thought about that. Humans losing their ability to detect AI content from reality ? It's frightening.
Oh they definitely are. A lot of people are now calling out real photos as fake. I frequently get into stupid Instagram political arguments and a lot of times they come back with "yeah nice profile with all your AI art haha". It's all real high quality photography. Honestly, I don't think the avg person can tell anymore.
I've reached a point where even if my first reaction to a photo is to be impressed, I then quickly think "oh but what it this is AI?" and then immediately my excitement for the photo is ruined because it may not actually be a photo at all.
Re: Why wordfreq will not be updated
#350I'm going to call it: The Web is dead. Thanks to "AI" I spend more time now digging through searches trying to find something useful than I did back in 2005. And the sites you do find are largely garbage. As a random example: just trying to find a particular popular set of wireless earbuds takes me at least 10 minutes, when I already know the company, the company's website, other vendors that sell the company's goods…
If search is your metric, the web was dead long before OpenAI's release of GPT. I gave up on web search a long time ago.