Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why wordfreq will not be updated

github.com

Post reply on HN
Print view View original

521–530 of 542 posts

Re: Why wordfreq will not be updated

#521

Post by derefr »

travisjungroth wrote: post #445
derefr wrote: post #331

Earlier quoted context omitted.

1. People don't generally use the (big, whole-web-corpus-trained) general-purpose LLM base-models to generate bot slop for the web. Paying per API call to generate that kind of stuff would be far too expensive; it'd be like paying for eStamps to send spam email. Spambot developers use smaller open-source models, trained on much smaller corpuses, sized and quantized to generate text that's "just good enough" to pass m…

On point 1, that’s surprising to me. A 2,000 word blog post would be 10 cents with GPT-4o. So you put out 1,000 of them, which is a lot, for $100.

There are two costs associated with using a hosted inference platform: the OpEx of API calls, and the CapEx of setting up an account in the first place. This second cost is usually trivial, as it just requires things any regular person already has: an SSO account, a phone number for KYC, etc.

But, insofar as your use-case is against the TOUs of the big proprietary inference platforms, this second cost quickly swamps the first cost. They keep banning you, and you keep having to buy new dark-web credentials to come back.

Given this, it’s a lot cheaper and more reliable — you might summarize these as “more predictable costs” — to design a system around a substrate whose “immune system” won’t constantly be trying to kill the system. Which means either your own hardware, or a “being your own model” inference platform like RunPod/Vast/etc.

(Now consider that there are a bunch of fly-by-night BYO-model hosted inference platforms, that are charging unsustainable flat-rate subscription prices for use of their hardware. Why do these exist? Should be obvious now, given the facts already laid out: these are people doing TOU-violating things who decided to build their own cluster for doing them… and then realized that they had spare capacity on that cluster that they could sell.)

Re: Why wordfreq will not be updated

#522

Post by shmageggy »

shreddit wrote: post #478
sergiotapia wrote: post #252

Earlier quoted context omitted.

reading stuff like this makes me so happy. no matter how fucked up something may be there is always a way to clean right up.

I wouldn't be so optimistic. The thing you call "way" is actually just time. Yes, anything humanity does (good or bad) will fade with time. But do we have the amount of time to clean up X (and i don't refer to X as in "formally twitter")?

This is (one of the many) reasons why I care primarily about biodiversity and preventing as many human-caused extinctions as we can. Those are a permanent loss to the beauty and complexity of the universe built up over millions of years, and they are permanent and irreversible.

Re: Why wordfreq will not be updated

#523

Post by whimsicalism »

coldpie wrote: post #518
whimsicalism wrote: post #363

Earlier quoted context omitted.

this is not a gated community at all

True, that is maybe too strong a phrase, but I think it's close to accurate. I think the culture & medium provide kind of a self-selecting gate: it's just plain text and links to articles, with the discussion expected by culture to be fairly serious. I think that turns off enough people that it kind of forms its own gate shutting out the people that make "eternal Septembers" happen. But yeah, ultimately, you're right…

I think that HN eternal september'd at the beginning of the pandemic and the Reddit Apollo stuff was the nail in the coffin.

Re: Why wordfreq will not be updated

#524

Post by krapp »

coldpie wrote: post #518
whimsicalism wrote: post #363

Earlier quoted context omitted.

this is not a gated community at all

True, that is maybe too strong a phrase, but I think it's close to accurate. I think the culture & medium provide kind of a self-selecting gate: it's just plain text and links to articles, with the discussion expected by culture to be fairly serious. I think that turns off enough people that it kind of forms its own gate shutting out the people that make "eternal Septembers" happen. But yeah, ultimately, you're right…

The people who make Eternal Septembers happen show up here all the time. They show up here from Reddit and 4chan and elsewhere expecting this forum to be a free-for-all because the word "hacker" is in the title, and are inevitably shocked and disappointed to learn how aggressively moderated and tone-policed this place is.

They're the ones who post racist diatribes under green alt account and anti-vaxx conspiracy theories and complain about wokeness and talk out of their ass about subjects they have no actual expertise in. The ones who think flagging is just a super downvote.

I mean, the premise that simply having a minimalist layout would filter for a better class of people has always been a bit silly. Eternal September is an inevitable result of having new people join a community. Every new person, every new comment, increases entropy. It doesn't matter how technically proficient they are, they will change the culture simply by virtue of their existence and participation at scale.

Re: Why wordfreq will not be updated

#525

Post by travisjungroth »

derefr wrote: post #521
travisjungroth wrote: post #445

Earlier quoted context omitted.

On point 1, that’s surprising to me. A 2,000 word blog post would be 10 cents with GPT-4o. So you put out 1,000 of them, which is a lot, for $100.

There are two costs associated with using a hosted inference platform: the OpEx of API calls, and the CapEx of setting up an account in the first place. This second cost is usually trivial, as it just requires things any regular person already has: an SSO account, a phone number for KYC, etc. But, insofar as your use-case is against the TOUs of the big proprietary inference platforms, this second cost quickly swamps…

This makes sense. But now I’m wondering if people here are speaking from experience or reasoning their way into it. Like are there direct reports of which models people are using for blogspam, or is it just what seems rational?

Re: Why wordfreq will not be updated

#526

Post by travisb »

Eisenstein wrote: post #429
travisb wrote: post #425

Earlier quoted context omitted.

Uh yeah. More tolerance is pretty much the definition of the social-left.

Are you saying that the Overton window can never move right as long as we maintain more human rights and tolerance than we had decades before whatever the current time period is?

No. I'm saying once you smooth over five or ten year periods, that it is empirically false that the Overton window in North America is moving rightwards.

It may very well feel that way to people who live in strong bubbles, but it just isn't true across the general population -- which of course is how the Overton Window is defined.

Re: Why wordfreq will not be updated

#527

Post by frognumber »

MengerSponge wrote: post #361
frognumber wrote: post #142

Earlier quoted context omitted.

There are a series of challenges like: https://www.nytimes.com/interactive/2024/09/09/technology/ai... https://www.nytimes.com/interactive/2024/01/19/technology/ar... These are a little bit unfair, in that we're comparing handpicked examples, but I don't think many experts will pass a test like this. Technology only moves forward (and seemingly, at an accelerating pace). What's a little shocking to me is the speed of…

Democracy is 200? You're off by a full order of magnitude. Progress isn't inevitable. It's possible for knowledge to be lost and for civilization to regress.

Okay. You're right about what I wrote. Let me rephrase what I meant. I was missing the words "the widespread adoption of"

Athens had a democracy over 2500 years ago. A few Native American tribes had long-lasting democracies. Ukrainian cities were democratically self-governing 500 years ago, and Poland had elected kings.

Those were isolated examples. This was not a revolution. We also haven't regressed; isolated examples continued throughout history. If you point to a year, you can probably find some democracy somewhere. The only major regression I know in history was around 1000BC. Regressions are rare.

What changed was a revolution. From just before 1800 to just a little after 1900, virtually every country had a revolution which led to either being some form of democracy, or pretending to be one. Democracy was no longer isolated. We had the creation of a free world covering much of the world's population, and the creation of what was pretending to be a democracy (today, even the Democratic People's Republic of Korea pretends to be a democracy).

The number of countries which claim to not be a democracy, you can count on your fingers. Iran. Vatican City. Saudi Arabia. UAE. Oman. Eswanti. Did I miss any?

https://en.wikipedia.org/wiki/List_of_countries_by_system_of...

Re: Why wordfreq will not be updated

#528

Post by Eisenstein »

travisb wrote: post #526
Eisenstein wrote: post #429

Earlier quoted context omitted.

Are you saying that the Overton window can never move right as long as we maintain more human rights and tolerance than we had decades before whatever the current time period is?

No. I'm saying once you smooth over five or ten year periods, that it is empirically false that the Overton window in North America is moving rightwards. It may very well feel that way to people who live in strong bubbles, but it just isn't true across the general population -- which of course is how the Overton Window is defined.

How does 'moderate positions now would be extreme in the 90s' act to bolster that contention, without the unsaid requirement that 'any included tolerance or added human rights that didn't exist more than a decade ago means de-facto left-wing Overton movement', which precludes any democratic society from having the Overton move right?

Re: Why wordfreq will not be updated

#529

Post by linhns »

pavel_lishin wrote: post #97
Retr0id wrote: post #78

Earlier quoted context omitted.

Arguably it is a form of language evolution. I bet humans have started using "delve" more too, on average. I think the best we can do is look at the trends and think about potential causes.

> I bet humans have started using "delve" more too, on average. I wish there were a way to check.

I'm seeing more and more of uses of it on this thread.

Re: Why wordfreq will not be updated

#530

Post by cdrini »

advael wrote: post #512
cdrini wrote: post #502

Earlier quoted context omitted.

Haha I'm pleasantly surprised to see my comment at the top, I genuinely thought it would drown to the bottom! Not due to disagreement, just due to sheer volume and being posted rather late in this posts lifespan. Anyways my meta comment wasn't that I disagreed with all the other comments, I was just frustrated at how repetitive they were of one another. When I go to leave a comment, I do a pass reading through all or…

The most ridiculous aspects for me were the anthropomorphizing (Reminds me of that one Sam Altman interview a bit) and the use of "infinite", which both doesn't really work on vibes (as many have noted, while I'm sure chatGPT has been exposed to every word, its pattern of communication is very "regression to the mean" among them), but also is silly if taken literally, because unless we're counting like some quirky te…

Ah ok so anthropomorphizing and the phrase "infinite vocabulary" sounds impossible. I agree infinite vocabulary is a bit murky, and mathematically incorrect. If I wanted to be more mathematically correct I could say complete vocabulary, but I think that's actually a little less understandable to people. I did not mean infinite vocabulary in that it coins new words, just infinite as in very large to the point of being incomprehensibly large by a single individual. As per anthropomorphizing, I think the word "chat" is the most anthropomorphizing I did, so don't agree with you on that one.

Ah mate sorry to hear that, the market is tough right now. I will say objectively I believe there's very little in my comment that's hype-y. I think using AI while reading documents out of your comfort zone, and asking it questions can expand your vocabulary. I've personally tried it, it's helped me read papers not in my field, it's helped me find papers for better research. I can understand how someone can disagree with that, but calling it hype sounds to me more like a response to an invisible enemy/to "all the ones who hyped before" than to an actual concrete response to this specific case. And I think that mentality could put you in a potential catch-22 mental loop that will leave you constantly dissatisfied with anything AI or ML, by constantly seeing this invisible enemy where it might not be present. Anyways, stay positive and best of luck with the job hunt!

Edit: and it looks like my comment has now fallen deep into the depths of the comment thread, never to be heard from again! See, I told you I was an embattled ideological minority ;)

Post reply on HN