Live data from Hacker News

2.1M of the oldest Usenet posts are now online for anyone to read

vice.com

31–40 of 320 posts

Re: 2.1M of the oldest Usenet posts are now online for anyone to read

#31

Can someone explain Usenet to me? As I understand it, it's not a website. Is it something outside of the WWW? Does it have its own protocol or does it use HTTP? Do you need a dedicated client software for it, similar to IRC? How do/did you register to it? Is it paid? Who hosts it?

Yes. It's outside the WWW. Originally, it was outside the Internet.

The basic concept is simply copying and forwarding email-like messages, grouped into a hierarchical set of newsgroup feeds. A host would subscribe to other hosts, and would allow other hosts to subscribe to their own feeds. And then just push/pull this through the network.

Originally, this was done all with shell scripts dialing up remote hosts by modem and copying compressed datasets nightly. This developed into a full suite of protocols called UUCP. Later, Internet protocols like the Network News Transfer Protocol started carrying some, then all, of the traffic.

In principle, it is unregistered, unpaid, and decentralized. You start exchanging with someone else who has the newsgroups you want to send and receive.

In practice, both in the past and now, you subscribe, usually commercially, to one of the services with the bandwidth and storage to host a reasonable portion of Usenet. ISPs often provided NNTP Usenet service into the early 2000s, along with SMTP/POP for email.

Re: 2.1M of the oldest Usenet posts are now online for anyone to read

#32
post #20

I and a number of people I know are grateful to Henry Spencer for not saving certain groups from the early days. On groups like soc.singles, we were young, lonely, indiscreet and posted using our real names, because it was a private club then (mid-80s).

Oh boy, thank God people my age don't have to worry about the incredibly embarrassing things we did, or embarrassing fanfiction we wrote, preserved publicly forever.

Gen Z, woooooo. Woo.

I'm doomed.

Re: 2.1M of the oldest Usenet posts are now online for anyone to read

#33

Can someone explain Usenet to me? As I understand it, it's not a website. Is it something outside of the WWW? Does it have its own protocol or does it use HTTP? Do you need a dedicated client software for it, similar to IRC? How do/did you register to it? Is it paid? Who hosts it?

Not only was it outside of the WWW, it predated the existence of the WWW by a good 15 years.

Re: 2.1M of the oldest Usenet posts are now online for anyone to read

#34
post #15
post #2

Wasn't this on Deja News that Google bought and integrated into Google groups? How is this archive different?

It really seems to me that archive.org, or some similar organisation, should petition Google to donate their Usenet archive, which they clearly don't care about, to them. Or possibly buy it off them. It's a long shot since it would require Google to actually do something with a dataset they clearly don't care about, but they could get some good corporate PR points almost for free.

Assuming they haven’t long ago given up on any kind of corporate PR points.

Re: 2.1M of the oldest Usenet posts are now online for anyone to read

#35

Can someone explain Usenet to me? As I understand it, it's not a website. Is it something outside of the WWW? Does it have its own protocol or does it use HTTP? Do you need a dedicated client software for it, similar to IRC? How do/did you register to it? Is it paid? Who hosts it?

Think something like Reddit + Bittorrent.

Slight difference -- there would be no main "front page," -- only individual self-hosted hubs that could choose which "subreddits" (newsgroups) they would carry.

Re: 2.1M of the oldest Usenet posts are now online for anyone to read

#36
post #15
post #2

Wasn't this on Deja News that Google bought and integrated into Google groups? How is this archive different?

It really seems to me that archive.org, or some similar organisation, should petition Google to donate their Usenet archive, which they clearly don't care about, to them. Or possibly buy it off them. It's a long shot since it would require Google to actually do something with a dataset they clearly don't care about, but they could get some good corporate PR points almost for free.

archive.org has various usenet archives. Some of the data is there, it's just not conveniently organized or searchable or viewable.

eg:

https://archive.org/details/usenet

Re: 2.1M of the oldest Usenet posts are now online for anyone to read

#37
post #27
post #12

I'm a bit surprised that a zip archive of only 2.1 million posts was almost 2GB. I wonder what a treemap of group size looks like.

When you factor in that ZIP compresses by about half, that averages to around 2K per post -- one screenful of text on an 80x25 PC display -- which is unsurprising. It's probably distributed in terms of a few long, informative posts and a long tail of "AOL!" type posts, but still.

Well I just noticed that fully one percent of these posts are in alt.atheism. I wonder how much this influence by UI design and lexical ordering alone. I bet alt.atheism was always on the first page when you just wanted to flip through groups.

Re: 2.1M of the oldest Usenet posts are now online for anyone to read

#38

Can someone explain Usenet to me? As I understand it, it's not a website. Is it something outside of the WWW? Does it have its own protocol or does it use HTTP? Do you need a dedicated client software for it, similar to IRC? How do/did you register to it? Is it paid? Who hosts it?

An important part about usegroups is that it is from an era of dial-in connections. One can dial into your ISP, sync News (fetch new messages in groups one is subscribed to and send messages) and then go offline and read/answer the messages without paying per minute and without blocking ports for other dial-in customers. This also worked between different news servers in distinct networks with unreliable connections.…

I see. It seems somewhat like mailing lists to me. I go online, my client sends and receives messages depending on which groups I'm subscribed to, and messages bounce around in a distributed fashion, somewhat like emails bouncing between SMTP servers.

I still don't quite get the user experience. What if you're subscribed to alt.binaries? Do you then sync all the binaries when you go online? That wouldn't make sense. Perhaps it only synchronizes on a level below that, alt.binaries.something.something? But that may still be too much.

And what if my ISP's server didn't have the newsgroup I was interested in? Is it like DNS, where it would then fetch it from somewhere else and then cache it? Or did most servers store the entire data? Or an ISP would just store the messages that at least one of their customers subscribe to?

Re: 2.1M of the oldest Usenet posts are now online for anyone to read

#39
post #4

Site: https://www.usenetarchives.com/

Though, the added 2.1M posts seems to be specifically among the ones here: https://usenetarchives.com/groups.php?c=utzoo

Thanks! Some great stuff under there, including the ranting review of the first episode of Star Trek: The Next Generation on TV from 1987: https://usenetarchives.com/view.php?id=alt.flame&g=224&p=0

Amazingly, you could replace most of the content of that post with "Star Trek Discovery" or "Star Trek Picard" and it'd be just like a recent post!

I think The Next Generation fared well despite the slow first series. Fascinating bit of internet history.

At least this "review" is worded better than many of the suspect (paid for?) reviews of films on IMDB these days.

Post reply on HN