Live data from Hacker News

Twitter now requires an account to view tweets

techcrunch.com

691–700 of 1001 posts

Re: Twitter now requires an account to view tweets

#691
post #618

Earlier quoted context omitted.

Well they killed the API, what did they think would happen? It's easier to control access and rate limit with a proper API.

The actual problem seems to be that a large number entities now want a full copy of the entire site. But why not just... provide it? Charge however much for a box of hard drives containing every publicly-available tweet, mailed to the address of buyer's choosing. Then the startups get their stupid tweets and you don't have any load problems on your servers.

What do you even charge for that? We might never make a repository of human made content with no Ai postings in it ever again. Seems like selling the golden goose to me

Re: Twitter now requires an account to view tweets

#692
post #650
post #110

This killed nitter. Fuck. I guess I'm done with Twitter. Reddit is in Eternal September. Twitter is login-walled. If HN is next, I'll probably be mostly done with the Internet. This version of the Internet is starting to suck. :(

HN isn't next. We hate change. (More precisely: we're acutely aware that users hate change, and since we do too, it's kind of an easy call.) Also, the value of HN to YC consists of the community and keeping the community happy is therefore a must. (Did I say happy? More precisely: as happy as possible under the circumstances)

Elon says he used it as a way to stop AI data scraping because the servers were suddenly hit by a large load.... (As a weird form of DDOS shielding in other words)

HN could be hit with such a large load too, since we have pretty good and lengthy discussion here, good for AI data training.

Would you believe it a valid tool to keep the community happy?

Since I doubt we would be happy if we can't access the site because some AI decided this was the time to scrape, either.

Re: Twitter now requires an account to view tweets

#693

Earlier quoted context omitted.

> This version of the Internet is starting to suck. :( The internet is fine. The highly centralized businesses that built themselves on the technology are meeting their inevitable end.

Wait until you realize that highly centralized businesses are a feature, not a bug. We've BEEN through federated platforms before. We've even been through PROTOCOLS before. They're all horrible. The successor to any platform that currently exists will have slight improvements to what already exists, and that's IF they're able to do so. I don't have a dog in this fight, but I do have over 30 years of being around soci…

Every time someone says this they forget the internet itself is a federated platform.

Re: Twitter now requires an account to view tweets

#694

I've viewed twitter wayyyyy less (only when someone links something) since Tweetbot stopped working and I think it's been good for my mental health. A login wall will take that usage down to zero, so good news overall.

I've gone from spending 5-10 hours a week on twitter to less than 30 minutes a week. Keep working your magic Elon.

Re: Twitter now requires an account to view tweets

#695
post #638
post #534

Earlier quoted context omitted.

It never seemed right that a private company was being put in between public functions and the the public which depends on them. But why? It’s always been that way. Before Twitter, it was private radio stations, private TV stations or private newspapers.

Many countries have TV and Radio channels that are funded by the state. The UK has BBC, for example.

BBC is not state funded.

Re: Twitter now requires an account to view tweets

#696
post #331

I've viewed twitter wayyyyy less (only when someone links something) since Tweetbot stopped working and I think it's been good for my mental health. A login wall will take that usage down to zero, so good news overall.

I find Twitter really annoying to use now, since they made it so all of the replies from blue checkmark accounts get boosted to the top of the feed. I have to scroll past all of them just to find "normal" comments. The BlueBlocker browser extension is helping, but it can only block 1 account approximately every 15 seconds, or it triggers a rate limit and I have to sign-in to Twitter again.

Yeah the replies were always the best part to me. That's destroyed now along with most of my interest in twitter.

Re: Twitter now requires an account to view tweets

#697

Earlier quoted context omitted.

At that point “just” normalize RSS feeds. They serve an even better job for none of the centralization. My state already sends weather warnings via RSS, and I’m sure lots of other things I haven’t explored yet.

No ad revenue can be made off RSS. Google made sure of that.

A government is not making ad revenue.

Re: Twitter now requires an account to view tweets

#698
post #650
post #110

This killed nitter. Fuck. I guess I'm done with Twitter. Reddit is in Eternal September. Twitter is login-walled. If HN is next, I'll probably be mostly done with the Internet. This version of the Internet is starting to suck. :(

HN isn't next. We hate change. (More precisely: we're acutely aware that users hate change, and since we do too, it's kind of an easy call.) Also, the value of HN to YC consists of the community and keeping the community happy is therefore a must. (Did I say happy? More precisely: as happy as possible under the circumstances)

I appreciate that attitude and value it myself, but I like to point out that it is not without risk.

If the world around HN (including its community) changes, stasis can damage or kill it as well.

Specifically regarding the issue of the original posting:

- HN is already an important data source for large language model training. [1]

- To the best of my knowledge there is no freely downloadable and current data dump of HN. [2]

- The HN-API does not offer all the data that scraping can get. For example, if a post had ever hit the front page or the highest front page position reached, is an interesting data point that is missing.

- The Algolia-HN-API has the same limitations.

In my opinion this will lead to increased usage of the API and increased scraping which all costs money. HN might be forced to find a solution for this.

[1] For example, the RefinedWeb paper lists HN as one of only 12 websites that were excluded. From what I understand, it was excluded because it went into the final dataset unvetted. RefinedWeb was used for the Falcon model.

https://arxiv.org/pdf/2306.01116.pdf

[2] The closest thing is probably the Google BigQuery "bigquery-public-data.hacker_news" dataset. It claims to be updated daily, but really is from late September 2022. Also I could not find the download link which other data sets offered on BigQuery have. Does anyone know if I can download the complete thing anyhow?

https://console.cloud.google.com/bigquery?p=bigquery-public-...

Re: Twitter now requires an account to view tweets

#699

Earlier quoted context omitted.

The actual problem seems to be that a large number entities now want a full copy of the entire site. But why not just... provide it? Charge however much for a box of hard drives containing every publicly-available tweet, mailed to the address of buyer's choosing. Then the startups get their stupid tweets and you don't have any load problems on your servers.

What do you even charge for that? We might never make a repository of human made content with no Ai postings in it ever again. Seems like selling the golden goose to me

> We might never make a repository of human made content with no Ai postings in it ever again.

Wow, never thought of it that way before. Kinda hit me hard for some reason.

Post reply on HN