Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Newsgroups and the Internet Archive: I Made a Difference

screengod.blogspot.com

Post reply on HN
Print view View original

11–20 of 65 posts

Re: Newsgroups and the Internet Archive: I Made a Difference

#11

Post by Steuard »

It's great to see some Usenet archives out there to partly make up for the disappointment of Google Groups. But I'm sad that this archive seems to be incomplete, even within its stated date range. Back in the day, I was active on rec.arts.books.tolkien and alt.fan.tolkien: in this archive, I can't find any trace of the massive "alt." hierarchy at all, and the list of files for the "rec." hierarchy doesn't include the Tolkien group. For that matter, the list includes rec.humor.funny and rec.humor.d and others, but apparently not rec.humor itself. (It really does make you appreciate just how substantial the effort of collecting a comprehensive Usenet archive would be.)

On another note, not that anyone here would be able to fix it, but this list would be a lot easier to search through if the item names didn't all begin with "Usenet newsgroups within", so you could jump to first letters in a meaningful way.

Re: Newsgroups and the Internet Archive: I Made a Difference

#12

Post by mailxplorer »

Steuard wrote: post #11

It's great to see some Usenet archives out there to partly make up for the disappointment of Google Groups. But I'm sad that this archive seems to be incomplete, even within its stated date range. Back in the day, I was active on rec.arts.books.tolkien and alt.fan.tolkien: in this archive, I can't find any trace of the massive "alt." hierarchy at all, and the list of files for the "rec." hierarchy doesn't include the…

It's my fault (not the IA), I thought I'd got all newsgroups but must have missed some. I just checked the main newsgroup list and it's incomplete, for some reason.

My plan though, is to dust off the old code, get a complete list of groups, get them, and then make it searchable.

Sorry about that, I didn't check. This was all done in 2013. Basically, I wanted to build a search engine but indexing the newsgroup posts (for header and text body search) would take too long. I abandoned it, then in 2016 I sent it to the IA.

So, I only just found out it's lacking a bunch of groups, 5 years on...

Re: Newsgroups and the Internet Archive: I Made a Difference

#13

Post by NelsonMinar »

Google's Usenet archive is too important to be owned by one company. My understanding is some Google engineers worked a few years ago to make sure Archive had a copy, but I can't verify that and I don't think it's online. Some of the earliest archives come from Eugene Spafford's collection which is readily online elsewhere, but Google did a lot of work cleaning it and of course has the DejaNews archives which are invaluable.

I'm amazed Google Groups still exists as a product, it seems abandoned internally and I expect to hear it's shut down any month now.

Re: Newsgroups and the Internet Archive: I Made a Difference

#14

Post by NelsonMinar »

luckydude wrote: post #2

I miss dejanews - anyone remember that before Google bought them? They had full search, it's a shame Google let that go. Their search went back to pretty much the beginning of net news. Does that exist anywhere anymore?

At least some of the Usenet archives are still online in Google Groups and searchable. Can't vouch for completeness of the search index though, it looks pretty wonky for stuff from the early 1990s. Anyway, example working archive link I found via search: https://groups.google.com/forum/#!searchin/comp.os.minix/tor...

Don't be too hard on Google's acquisition of Deja. I wasn't working at Google at the time, but heard from many colleagues when I joined that the Deja acquisition was quite chaotic because Deja was just about to shut down entirely when Google picked it up rather than let it disappear. It's a shame they don't do better with the Usenet archive now, but it's clearly not Google's business anymore.

Re: Newsgroups and the Internet Archive: I Made a Difference

#15

Post by bhaak »

Steuard wrote: post #11

It's great to see some Usenet archives out there to partly make up for the disappointment of Google Groups. But I'm sad that this archive seems to be incomplete, even within its stated date range. Back in the day, I was active on rec.arts.books.tolkien and alt.fan.tolkien: in this archive, I can't find any trace of the massive "alt." hierarchy at all, and the list of files for the "rec." hierarchy doesn't include the…

It looks like the downloads are grouped by top level group.

So you can go to https://archive.org/download/gna-rec and see all archived groups that are in the rec hierarchy.

Re: Newsgroups and the Internet Archive: I Made a Difference

#16

Post by nikanj »

mmanfrin wrote: post #10
naasking wrote: post #8

Earlier quoted context omitted.

Google search in general has become less effective, so perhaps it simply carried over.

Search is abysmal these days. I'll look for a term with 3 words and it almost always throws out one word, giving me useless results.

”You searched for Stockholm Syndrome Research. Here’s a bunch of Stockholm travel blogs! They’re just missing two of your keywords, but they’re really popular with other visitors”

Re: Newsgroups and the Internet Archive: I Made a Difference

#17

Post by cultureleak_ta0 »

ghaff wrote: post #9
Steuard wrote: post #5

Earlier quoted context omitted.

Google Groups was great when they first acquired all those archives! It just got progressively worse and worse over time, and I've never understood why. (Stagnation I could understand, but how did the same searches get less effective?)

I get that Google essentially walked back from an at least implied role as an information archivist. But I still don't really get why they so completely abandoned things like Google Groups given the truly minuscule resources to maintain them at some low level.

Nobody can get promoted for improvements like that. It would only garner goodwill with a tiny customer base and it’s not clear how that would translate to revenue or user growth on other products

Re: Newsgroups and the Internet Archive: I Made a Difference

#18

Post by wmf »

cultureleak_ta0 wrote: post #17
ghaff wrote: post #9

Earlier quoted context omitted.

I get that Google essentially walked back from an at least implied role as an information archivist. But I still don't really get why they so completely abandoned things like Google Groups given the truly minuscule resources to maintain them at some low level.

Nobody can get promoted for improvements like that. It would only garner goodwill with a tiny customer base and it’s not clear how that would translate to revenue or user growth on other products

But there do seem to be quite a few people at Google already working on random stuff that won't get them promoted.

Re: Newsgroups and the Internet Archive: I Made a Difference

#19

Post by digi_owl »

ghaff wrote: post #9
Steuard wrote: post #5

Earlier quoted context omitted.

Google Groups was great when they first acquired all those archives! It just got progressively worse and worse over time, and I've never understood why. (Stagnation I could understand, but how did the same searches get less effective?)

I get that Google essentially walked back from an at least implied role as an information archivist. But I still don't really get why they so completely abandoned things like Google Groups given the truly minuscule resources to maintain them at some low level.

What seems to have happened was that Google shifted focus from sorting the web to delivering the "zeitgeist" of the present.

Perhaps because more people were likely to search for whatever was on the news cycle than some arbitrary piece of info.

Never mind that Google's earnings comes from ads, not search results.

Re: Newsgroups and the Internet Archive: I Made a Difference

#20

Post by ghaff »

digi_owl wrote: post #19
ghaff wrote: post #9

Earlier quoted context omitted.

I get that Google essentially walked back from an at least implied role as an information archivist. But I still don't really get why they so completely abandoned things like Google Groups given the truly minuscule resources to maintain them at some low level.

What seems to have happened was that Google shifted focus from sorting the web to delivering the "zeitgeist" of the present. Perhaps because more people were likely to search for whatever was on the news cycle than some arbitrary piece of info. Never mind that Google's earnings comes from ads, not search results.

Google's original mission statement from 1998 was to “organize the world’s information and make it universally accessible and useful.” Things like Usenet archives are clearly part of that.

That is still apparently their mission statement but I agree the reality seems more along the lines of "deliver the best search results for users' immediate needs" [while making as much money from that as humanly possible]

Post reply on HN