Live data from Hacker News

An update on Wayback Machine access

blog.archive.org

131–140 of 374 posts

Re: An update on Wayback Machine access

#131
post #8
post #6

Earlier quoted context omitted.

That's for the lawyers to sort out, they have a lot of flexibility as a US nonprofit. The case law isn't that clear cut for this.

Internet Archive was almost destroyed by a copyright lawsuit from book publishers within the last few years. I expect they aren't excited to take on any additional risk of similar lawsuits right now.

They brought it on themselves by marketing a read-for-free product

Re: An update on Wayback Machine access

#132

Earlier quoted context omitted.

I assume that's because the IP range of your company network overlaps with a range used by some scrapers, and if it doesn't happen on your phone even in the corporate network, then IA probably checks some extra signals like the user agent in addition to the IP

No - my phone is not connected to work's WiFi. Wonder who the bad actors in my company are...

Doesn't have to be someone at your work, it could be a block on a whole ISP network or at least an IP subnetwork that is shared between many clients

You can try emailing the address mentioned in their post so they adjust their filters to match just the bot networks more precisely

Re: An update on Wayback Machine access

#133

Earlier quoted context omitted.

I assume that's because the IP range of your company network overlaps with a range used by some scrapers, and if it doesn't happen on your phone even in the corporate network, then IA probably checks some extra signals like the user agent in addition to the IP

No - my phone is not connected to work's WiFi. Wonder who the bad actors in my company are...

You should file a support ticket with your manager and the IT security or support desk. Show them the evidence of 429s that are blocking your assigned tasks during working hours. Also include the screenshots and files that you downloaded on your personal device in order to access your work-related materials. Be sure and thank them for adequately configuring the MDM on your personal mobile device so that you could do these work-related tasks. You should definitely also file an expense report to request reimbursement of your personal mobile bill, any data charges incurred, and the hours of networking or collaborating with external colleagues, while you were working on these work-related projects with your personal device.

Re: An update on Wayback Machine access

#134

Earlier quoted context omitted.

I'd pay for it, but only if they implemented the changes the community of users have been requesting for years.

No matter what they did, you'd have a new excuse for why you won't pay.

Ah, the old ad hominem attack. How refreshing.

But anyway, no, I wouldn't keep finding reasons. I donate to them every year already. Somebody asked if I would be willing to pay and my answer was "yes, but".

It would need to be improved because certain aspects of it suck right now, not only the error this post is about. They only need go as far as their forums and github repos to see the community feedback.

Re: An update on Wayback Machine access

#135
post #109

Earlier quoted context omitted.

They get marked as inaccessible, but still exist in IA data

is it possible to access somehow? it seems the site got excluded because of robots.txt set by some domain squatter, not manually.

Get a job at IA?

Re: An update on Wayback Machine access

#138
post #97

I've been getting this error a lot. Asking users to email them with details of their OS, browser, IP address is just crazy. Their support is supposedly already swamped and they are asking for more!? Changes made by IA shouldn't become my responsibility.

> Asking users to email them with details of their OS, browser, IP address is just crazy. It's surely to serve as data to help tell humans apart from bots. > Changes made by IA shouldn't become my responsibility. They're a free service. It's ultimately not their responsibility to service you either.

Imagine if Apple or Microsoft introduced a bug and said, ah yes we know about it we did that on purpose and we know it affects a huge number of people, if each of you could email us these details that'd be great. It's just such an insane request.

IA have broken it and have no real idea how to make it better so they are going to whitelist IPs or browsers or entire operating systems? Wild.

Re: An update on Wayback Machine access

#139

Earlier quoted context omitted.

I suspect most people would be ok with this if they could only do it at the rate and frequency you yourself can do it. The problem is largely one of scale.

Then make agent friendly content. Take the text and make a markdown version.

People doing this say it makes things worse because then the bots download both.
Post reply on HN