Live data from Hacker News

ArchiveTeam has finished archiving all goo.gl short links

tracker.archiveteam.org

11–20 of 112 posts

Re: ArchiveTeam has finished archiving all goo.gl short links

#11
post #4

Earlier quoted context omitted.

They iterated the entire URL namespace by having volunteers run a client so they didn't get IP banned.

are we sure that the whole entire URL namespace has been mapped? How would that even function, I mean, did they loop through every single permutation and see the result, or what exactly/ how would that work?

The pipeline code is available for review of the mechanics of http requests made if you follow the ArchiveTeam wiki links.

Re: ArchiveTeam has finished archiving all goo.gl short links

#13
Related. Others?

Enlisting in the Fight Against Link Rot - https://news.ycombinator.com/item?id=44877021 - Aug 2025 (107 comments)

Google shifts goo.gl policy: Inactive links deactivated, active links preserved - https://news.ycombinator.com/item?id=44759918 - Aug 2025 (190 comments)

Google's shortened goo.gl links will stop working next month - https://news.ycombinator.com/item?id=44683481 - July 2025 (222 comments)

Google URL Shortener links will no longer be available - https://news.ycombinator.com/item?id=40998549 - July 2024 (49 comments)

Ask HN: Google is sunsetting goo.gl on 3/30. What will be your URL shortener? - https://news.ycombinator.com/item?id=19385433 - March 2019 (14 comments)

Tell HN: Goo.gl (Google link Shortener) is shutting down - https://news.ycombinator.com/item?id=16902752 - April 2018 (45 comments)

Google is shutting down its goo.gl URL shortening service - https://news.ycombinator.com/item?id=16722817 - March 2018 (56 comments)

Transitioning Google URL Shortener to Firebase Dynamic Links - https://news.ycombinator.com/item?id=16719272 - March 2018 (53 comments)

Re: ArchiveTeam has finished archiving all goo.gl short links

#15

Title is imprecise, it's Archiveteam.org, not Archive.org. The Internet Archive is providing free hosting, but the archival work was done by Archiveteam members.

What exactly is archiveteam's contribution? I don't fully understand.

Edit: Like they kinda seem like an unnecessary middle-man between the archive and archivee, but maybe I'm missing something.

Re: ArchiveTeam has finished archiving all goo.gl short links

#16
post #4

Earlier quoted context omitted.

They iterated the entire URL namespace by having volunteers run a client so they didn't get IP banned.

are we sure that the whole entire URL namespace has been mapped? How would that even function, I mean, did they loop through every single permutation and see the result, or what exactly/ how would that work?

> did they loop through every single permutation and see the result, or what exactly/ how would that work?

In short, yes. Since no one can make new links, it's a pre-defined space to search. They just requested every possible key, and recorded the answer, and then uploaded it to a shared database.

Re: ArchiveTeam has finished archiving all goo.gl short links

#17
post #7

Recent update from Google: https://blog.google/technology/developers/googl-link-shorten...

I have a question about this.

Per google, shortened links “won't work after August 25 and we recommend transitioning to another URL shortener if you haven’t already.”

Am I missing something, or doesn’t this basically obviate the entire gesture of keeping some links active? If your shortened link is embedded in a document somewhere and can’t be updated, google is about to break it, no?

Re: ArchiveTeam has finished archiving all goo.gl short links

#18
post #15

Title is imprecise, it's Archiveteam.org, not Archive.org. The Internet Archive is providing free hosting, but the archival work was done by Archiveteam members.

What exactly is archiveteam's contribution? I don't fully understand. Edit: Like they kinda seem like an unnecessary middle-man between the archive and archivee, but maybe I'm missing something.

ArchiveTeam delegates tasks to volunteers and themselves running the Archive Warrior VM, which does the actual archiving. The resultant archives are then centralized by ArchiveTeam and uploaded to the Internet Archive.

(Source: ran a Warrior)

Re: ArchiveTeam has finished archiving all goo.gl short links

#19
post #7

Recent update from Google: https://blog.google/technology/developers/googl-link-shorten...

I don't really understand this. Is it really that costly to keep the entire database if they're going to keep part of it?

I built a URL shortener years ago for fun. I don't have the resources that Google has, but I just hacked it together in Erlang using Riak KV and it did horizontally scale across at least three computer (I didn't have more at the time).

Unless I'm just super smart (I'm not), it's pretty easy to write a URL shortener as a key-value system, and pure key-value stuff is pretty easy to scale. I cannot imagine that isn't doing something as or more efficient than what I did.

Re: ArchiveTeam has finished archiving all goo.gl short links

#20
post #15

Title is imprecise, it's Archiveteam.org, not Archive.org. The Internet Archive is providing free hosting, but the archival work was done by Archiveteam members.

What exactly is archiveteam's contribution? I don't fully understand. Edit: Like they kinda seem like an unnecessary middle-man between the archive and archivee, but maybe I'm missing something.

They gathered up the links for processing, because Google doesn't just give a list of short links in use. So the links have to be brute-forcefully gathered first.
Post reply on HN