Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

2.1M of the oldest Usenet posts are now online for anyone to read

vice.com

Post reply on HN
Print view View original

61–70 of 320 posts

Re: 2.1M of the oldest Usenet posts are now online for anyone to read

#61

Post by jdeibele »

bonoboTP wrote: post #38
johannes1234321 wrote: post #29

Earlier quoted context omitted.

An important part about usegroups is that it is from an era of dial-in connections. One can dial into your ISP, sync News (fetch new messages in groups one is subscribed to and send messages) and then go offline and read/answer the messages without paying per minute and without blocking ports for other dial-in customers. This also worked between different news servers in distinct networks with unreliable connections.…

I see. It seems somewhat like mailing lists to me. I go online, my client sends and receives messages depending on which groups I'm subscribed to, and messages bounce around in a distributed fashion, somewhat like emails bouncing between SMTP servers. I still don't quite get the user experience. What if you're subscribed to alt.binaries? Do you then sync all the binaries when you go online? That wouldn't make sense.…

You would (typically) have a dumb terminal experience. You'd use your computer and modem to log into a remote system. You would use a newsreader to browse newsgroups. If there was something that you wanted, like binaries, they would be saved on the remote system. You'd then need to ftp them from that system to your own.

If you were one of the fortunate ones to have a hard-wired connection, well, that happened a lot faster.

Newsreaders were awesome for reading newsgroups. You were able to add someone or a topic to a kill file for one group or all groups.

As binaries got bigger and bigger, the retention time for them on the remote system got shorter and shorter. You might have 7 days of binaries and 90 days of everything else.

For a while some forum software would be accessible via NNTP which was awesome. It let more people use the forums but gave more experienced users much better tools.

Re: 2.1M of the oldest Usenet posts are now online for anyone to read

#62

Post by gwittel »

bonoboTP wrote: post #38
johannes1234321 wrote: post #29

Earlier quoted context omitted.

An important part about usegroups is that it is from an era of dial-in connections. One can dial into your ISP, sync News (fetch new messages in groups one is subscribed to and send messages) and then go offline and read/answer the messages without paying per minute and without blocking ports for other dial-in customers. This also worked between different news servers in distinct networks with unreliable connections.…

I see. It seems somewhat like mailing lists to me. I go online, my client sends and receives messages depending on which groups I'm subscribed to, and messages bounce around in a distributed fashion, somewhat like emails bouncing between SMTP servers. I still don't quite get the user experience. What if you're subscribed to alt.binaries? Do you then sync all the binaries when you go online? That wouldn't make sense.…

Sort of like mailing lists with a dedicated client so you can see everything in nice threaded fashion. You could explore new groups through the client as well. I never found the web based Usenet mirrors comparable to a dedicated client.

Typically you synced the equivalent of headers and bodies separately (for the groups you were interested in). This way you only grabbed what you wanted. For binary groups, people settled on patterns in subjects, etc so that clients could automate downloading of files (binaries had to be encoded and split into parts due to post length limits).

On regular text type newsgroups you might set your client to download headers+bodies in one shot for offline mode. With offline mode you would then just download what's new vs your cache. Then work against your local cache for reading/replying. Once online again, your batched replies could be sent.

If your ISP didn't have the newsgroup you wanted, you would have to point to another NTTP server. For a while there were public ones that anyone could use. Over time, those public ones went away or dropped problematic areas like alt.binaries.*. These days Usenet is available as a for-pay service.

Re: 2.1M of the oldest Usenet posts are now online for anyone to read

#64

Post by acdha »

bonoboTP wrote: post #38
johannes1234321 wrote: post #29

Earlier quoted context omitted.

An important part about usegroups is that it is from an era of dial-in connections. One can dial into your ISP, sync News (fetch new messages in groups one is subscribed to and send messages) and then go offline and read/answer the messages without paying per minute and without blocking ports for other dial-in customers. This also worked between different news servers in distinct networks with unreliable connections.…

I see. It seems somewhat like mailing lists to me. I go online, my client sends and receives messages depending on which groups I'm subscribed to, and messages bounce around in a distributed fashion, somewhat like emails bouncing between SMTP servers. I still don't quite get the user experience. What if you're subscribed to alt.binaries? Do you then sync all the binaries when you go online? That wouldn't make sense.…

You'd subscribe to specific groups and many clients were smart enough to do varying levels of selective sync: fetch all of the message headers but not the bodies over a certain size, only messages matching certain patterns, etc. It was definitely high-volume and could have been risky — imagine if the predecessors to e.g. 4chan decided to cross-post illegal content to a channel.

This was also where the term “spam” really got popularized (see https://en.wikipedia.org/wiki/Laurence_Canter_and_Martha_Sie...) so many things which we now take for granted were being tried at scale for the first time there. There was a [relatively weak] moderation system which could be used to restrict posting and there were bots which would retroactively cancel messages so you might not see so much junk if a group used such a bot and ruthlessly nuked spam.

Regarding ISPs, in many cases it worked like your last case: start storing anything that at least one client has asked for recently. This was a notorious source of disk space consumption if you ran a low-capacity server and then had someone fire up a client, request the world, and then stop reading it but your system was still downloading everything.

Re: 2.1M of the oldest Usenet posts are now online for anyone to read

#66

Post by ars »

bonoboTP wrote: post #43
jrm4 wrote: post #35

Earlier quoted context omitted.

Think something like Reddit + Bittorrent. Slight difference -- there would be no main "front page," -- only individual self-hosted hubs that could choose which "subreddits" (newsgroups) they would carry.

How did you get a username? What was its format? Something structured like an email address or just any string?

It was normally your email address, but you could put anything.

Typically the posting news server (i.e. your own news server) was responsible for validating that you used your real email address.

Originally there was no validation, but as spam rose validation became required.

News servers required you to login, originally they were open to anyone.

Re: 2.1M of the oldest Usenet posts are now online for anyone to read

#67

Post by ioseph »

It's really striking the relatively high quality of post content. Jumping into a group related to an interest of mine all the discussions are very engaging.

Assuming there's little in the way of moderation perhaps there's something to be said for small networks with a barrier to entry.

Re: 2.1M of the oldest Usenet posts are now online for anyone to read

#68

Post by aeyes »

bonoboTP wrote: post #54
retrac wrote: post #31

Earlier quoted context omitted.

Yes. It's outside the WWW. Originally, it was outside the Internet. The basic concept is simply copying and forwarding email-like messages, grouped into a hierarchical set of newsgroup feeds. A host would subscribe to other hosts, and would allow other hosts to subscribe to their own feeds. And then just push/pull this through the network. Originally, this was done all with shell scripts dialing up remote hosts by mo…

So basically there was a fully synced copy of the entire newsgroup contents for the groups I'm subscribed to? Like it would fetch all the new messages that arrived in alt.atheism and store it on my disk? But that surely wouldn't work on alt.binaries. Was there something like a cutoff at a certain size and you'd have to explicitly request the full content for some specific messages?

Using the NNTP protocol (which is very similar to SMTP) each message has two parts: Headers and content. Your client would usually only download message headers which contains information like author, date, title and message size.

Back when UUCP was still around there wasn't much binary data in groups. It took a while to develop algorithms to encode binary data (uuencode, base64), the protocol was text only.

Re: 2.1M of the oldest Usenet posts are now online for anyone to read

#70

Post by throwaway2048 »

bonoboTP wrote: post #54
retrac wrote: post #31

Earlier quoted context omitted.

Yes. It's outside the WWW. Originally, it was outside the Internet. The basic concept is simply copying and forwarding email-like messages, grouped into a hierarchical set of newsgroup feeds. A host would subscribe to other hosts, and would allow other hosts to subscribe to their own feeds. And then just push/pull this through the network. Originally, this was done all with shell scripts dialing up remote hosts by mo…

So basically there was a fully synced copy of the entire newsgroup contents for the groups I'm subscribed to? Like it would fetch all the new messages that arrived in alt.atheism and store it on my disk? But that surely wouldn't work on alt.binaries. Was there something like a cutoff at a certain size and you'd have to explicitly request the full content for some specific messages?

generally speaking only usenet servers themselves grab everything at all times and store it, subject to an retention time (say 1 year, messages older than this are deleted) most clients just grab headers, and download messages on access.
Post reply on HN