Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

ArchiveTeam has finished archiving all goo.gl short links

tracker.archiveteam.org

Post reply on HN
Print view View original

41–50 of 112 posts

Re: ArchiveTeam has finished archiving all goo.gl short links

#41

Post by OJFord »

nocoiner wrote: post #17
Ayesh wrote: post #7

Recent update from Google: https://blog.google/technology/developers/googl-link-shorten...

I have a question about this. Per google, shortened links “won't work after August 25 and we recommend transitioning to another URL shortener if you haven’t already.” Am I missing something, or doesn’t this basically obviate the entire gesture of keeping some links active? If your shortened link is embedded in a document somewhere and can’t be updated, google is about to break it, no?

About to break it if it didn't seem 'actively used' in late 2024, yes. But if your document was being frequently read and the link actively clicked, it'll (now) keep working.

But as I said in sibling comment to yours, I don't see the point of the distinction, why not just continue them all, surely the mostly unused ones are even cheaper to serve.

Re: ArchiveTeam has finished archiving all goo.gl short links

#43

Post by horseradish7k »

im3w1l wrote: post #15
zdimension wrote: post #3

Title is imprecise, it's Archiveteam.org, not Archive.org. The Internet Archive is providing free hosting, but the archival work was done by Archiveteam members.

What exactly is archiveteam's contribution? I don't fully understand. Edit: Like they kinda seem like an unnecessary middle-man between the archive and archivee, but maybe I'm missing something.

liability shield

Re: ArchiveTeam has finished archiving all goo.gl short links

#44

Post by horseradish7k »

mdaniel wrote: post #35
yreg wrote: post #29

I wonder how many of them lead to private YouTube videos, Google documents, etc.

I was going to be cheeky and say "well, now you can download them and search" but it seems it's "Access-restricted-item: true" for some reason, above and beyond being 10G a pop https://archive.org/details/archiveteam_googl_20250228144231... >

you'd have to rescrape them all from https://web.archive.org/cdx/search?url=goo.gl/* - they don't publish the whole datasets

Re: ArchiveTeam has finished archiving all goo.gl short links

#45

Post by iamacyborg »

creatonez wrote: post #28
im3w1l wrote: post #15

Earlier quoted context omitted.

What exactly is archiveteam's contribution? I don't fully understand. Edit: Like they kinda seem like an unnecessary middle-man between the archive and archivee, but maybe I'm missing something.

What ArchiveTeam mainly does is provide hand-made scripts to aggressively archive specific websites that are about to die, with a prioritization for things the community deems most endangered and most important. They provide a bot you can run to grab these scripts automatically and run them on your own hardware, to join the volunteer effort. This is in contrast to the Wayback Machine's builtin crawler, which is just…

Their MediaWiki tool was also invaluable in helping us fork the Path of Exile wiki from Fandom.

Re: ArchiveTeam has finished archiving all goo.gl short links

#46

Post by DaSHacka »

SilverElfin wrote: post #37

Is there anyone archiving all of reddit? Or twitter? I mean even if their terms have changed to not allow it.

> reddit

There used to be one such project (Pushshift), before the Reddit API change. You can download all the data and see all the info on the-eye, another datahoarder/preservationist group:

https://the-eye.eu/redarcs/

> twitter

Not that I know of, and you haven't even been able to archive tweets on the Wayback machine for YEARS.

Re: ArchiveTeam has finished archiving all goo.gl short links

#48

Post by manquer »

OJFord wrote: post #9
Ayesh wrote: post #7

Recent update from Google: https://blog.google/technology/developers/googl-link-shorten...

This leaves me wondering what the point is? What could it possibly cost to keep redirecting existing shortlinks that they consider unused/low activity already anyway ? (In addition to the higher activity ones parent link says they'll now continue to redirect.)

For a company also running a hosting service like GCP? nothing.

They already have plenty of unused compute /older hardware / CDN POPs, performant distributed data store and everything else possibly needed .

It would be cheaper than the free credits they giveaway just one startup to be on GCP.

I don’t think infra costs are a factor in a decision like this .

Re: ArchiveTeam has finished archiving all goo.gl short links

#49

Post by iJohnDoe »

Why? Did they ask anyone if it was okay? Anything sensitive at those links? Anything at those links people didn't want or need anymore? Maybe people thought those links were dead? Did Google provide a way to cancel those links first?

It's like when the GPT links were archived and publicly available that contained sensitive information.

Re: ArchiveTeam has finished archiving all goo.gl short links

#50

Post by tombert »

wtallis wrote: post #25
tombert wrote: post #19

Earlier quoted context omitted.

I built a URL shortener years ago for fun. I don't have the resources that Google has, but I just hacked it together in Erlang using Riak KV and it did horizontally scale across at least three computer (I didn't have more at the time). Unless I'm just super smart (I'm not), it's pretty easy to write a URL shortener as a key-value system, and pure key-value stuff is pretty easy to scale. I cannot imagine that isn't do…

Google also has the advantages that they now only need a read-only key-value store, and they know the frequency distribution for lookups. This is now the kind of problem many programmers would be happy to spend a weekend optimizing to get an average lookup time down to tens of nanoseconds.

I don't think it would even cost me very much to host all these links on a GCP or AWS thing, I don't think more than a couple hundred dollars a year.

Obviously raw server costs aren't the only costs associated with something like this, you'd still need to pay software people to keep it on life support, but considering how simple URL shorteners are to implement, I still don't think it would be that expensive.

ETA:

I should point out, even something kind of half-assed could be built with Cloud Functions and BigTable really easily; this wouldn't win any kind of contests for low latency, but it would be exceedingly simple code and have sufficient uptime guarantees and would be much less likely to piss off the community.

If I had any idea how to reach out to higher-ups at Google I would offer to contract and build it myself, but that's certainly not necessary, they have thousands of developers, most of which could write this themselves in an afternoon.

Post reply on HN