Viewing profile — TheTechRobo

Browsertrix (and Webrecorder tools in general) also violate the standard by modifying response data. It's supposed to be the raw bytes as they are sent over the network (minus TLS)…

Unfortunately the Raspberry Pi version isn't supported anymore because their version of Wget isn't well-tested on ARM. I'm surprised it's still there in the repo. You can get Docke…

That doesn't really confirm that, IMHO. The line before claims the opposite, in fact: "The Internet Archive has been around since 1996, and while that does not guarantee anything, …

Everything transmitted over HTTP. That includes cookies, passwords, etc. You need to be careful when writing WARCs.

Yeah, the format really needs an update. For starters, WARC only officially supports HTTP/1.1. Webrecorder has started faking HTTP/1.1 data in WARC files in order to save other ver…

ArchiveTeam's general idea is to archive it first, and make it usable later. Wayback Machine is generally how stuff is "made usable", but in cases like this, a searchable index mig…

Yes, what is your point?

Archive Team hasn't really done much yet - all they've done is an ArchiveBot job, which isn't getting anywhere because of DPReview's ratelimits, and some discovery scrapes. This is…

You select the project you run at any given moment. You can also select Archiveteam's Choice, which is usually the most urgent or the one with the most IPs needed, but if you do wa…

Similar: github.com/InternetArchive/warcprox

There's github.com/ArchiveTeam/grab-site, but unfortunately it's not maintained very well.