161–170 of 227 posts
Earlier quoted context omitted.
And they're curiously usually protected by Cloudflare.
Doubt cloudflare has anything to do with it. The operators most likely don't want to openly expose their website's ip addresses.
This is why I’ve gotten into the habit of maintaining my own WWW archive of sites I find interesting. Probably have around 1 TiB now, and One Of These Days I’d like to set my network up so it can serve arbitrary sites directly from local archive to revive any site I want. I have a `wget-mirror` shell function invoking wget with all the trimmings that takes care of 99% of sites. I’ll edit the full command into this co…
I would love that. I have a little for parameter version, but I feel yours is more tried and true.
This is why I’ve gotten into the habit of maintaining my own WWW archive of sites I find interesting. Probably have around 1 TiB now, and One Of These Days I’d like to set my network up so it can serve arbitrary sites directly from local archive to revive any site I want. I have a `wget-mirror` shell function invoking wget with all the trimmings that takes care of 99% of sites. I’ll edit the full command into this co…
> I’ll edit the full command into this comment when I get home if anybody else wants to start doing the same :) I would love that. I have a little for parameter version, but I feel yours is more tried and true.
This is why I’ve gotten into the habit of maintaining my own WWW archive of sites I find interesting. Probably have around 1 TiB now, and One Of These Days I’d like to set my network up so it can serve arbitrary sites directly from local archive to revive any site I want. I have a `wget-mirror` shell function invoking wget with all the trimmings that takes care of 99% of sites. I’ll edit the full command into this co…
Assume everyone is familiar with this project, dating back to 1996: https://en.wikipedia.org/wiki/WWWOFFLE https://ftp.netbsd.org/pub/pkgsrc/distfiles/wwwoffle-2.9j.tg... The way the www is going, it seems like downloading a copy of libgen, i.e., nonfiction books, and scimag, i.e., academic journals, via torrent, would be more valuable than archiving websites, in general. These primary sources are part of the materia…
Earlier quoted context omitted.
I find the idea of DDoS providers confusing. If someone tried to operate a service that can be abused easily to cause similar disruption in the physical world, the operation would be taken down quickly and the people behind it would probably end up in prison. But somehow the internet is still a lawless zone where crime is tolerated and everyone is out for themselves.
It used to be very rare for DDoS providers to publicly advertise their services, you kinda had to know a guy who knew a guy. If you put up a website offering this service the Good Guys of the Internet would track you down and get your provider to take you down, or that provider would in turn get disconnected from the internet. Now they hide behind Cloudflare who will refuse to turn over any information so that securi…
Most of the listed domain names are under US jurisdiction. That means the authorities should be able to take them down. If Cloudflare is found to have been knowingly enabling crime, it could face fines, and the CEO and other key people could end up in prison. The Cloudflare services have probably been paid using means that are under US jurisdiction. Those payment accounts can be closed and the people behind them tracked down and potentially charged with crimes.
Or at least that's how things work in the real world. The internet is still apparently too new for the authorities to understand how to deal with it.
This is why I’ve gotten into the habit of maintaining my own WWW archive of sites I find interesting. Probably have around 1 TiB now, and One Of These Days I’d like to set my network up so it can serve arbitrary sites directly from local archive to revive any site I want. I have a `wget-mirror` shell function invoking wget with all the trimmings that takes care of 99% of sites. I’ll edit the full command into this co…
Missed the edit window, but here's the command I use. Newlines added here for clarity. wget-mirror() { wget --mirror --convert-links --adjust-extension --page-requisites \ --no-parent --content-disposition --content-on-error \ --header="Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8" \ --user-agent="Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:129.0) Gecko/20100101 Firefox/129.0" \ --restrict…
I have encountered "GnuTLS: The TLS connection was non-properly terminated. Unable to establish SSL connection." multiple times, and retry options seem to be useless when that happens. Some searches suggest it could be related to tls handshake fragmentation, but nonetheless wget could retry if related options are used. Manual retry seems to download the missing URLs, otherwise mirroring jobs are randomly incomplete.
If anyone from the archive.org team read this: love the website, by the way. You've saved so much rare content its really awesome.
would a decentralized internet archive make sense? impossible to ddos.
Earlier quoted context omitted.
Cloudflare is not "an extortionist firm". It is a large tech company, where occasionally teams employ shitty sales tactics to meet their numbers, but generally provides a valuable service and acts reasonably ethically. There are open source tools to mitigate DDoS, but all of them will have some marginal cost to run, and they will all be significantly worse than Cloudflare as they benefit from neither Cloudflare's dat…
No, thanks. Cloudflare acts ethically only until it suits them. It is the pre-exploitation phase to lure a customer. We are not fools here. The report at https://news.ycombinator.com/item?id=40481808 says it all. Secondly, considering Cloudflare would MITM all traffic, it would make a data good source for the NSA, thereby violating all user privacy.
Earlier quoted context omitted.
Cloudflare is not "an extortionist firm". It is a large tech company, where occasionally teams employ shitty sales tactics to meet their numbers, but generally provides a valuable service and acts reasonably ethically. There are open source tools to mitigate DDoS, but all of them will have some marginal cost to run, and they will all be significantly worse than Cloudflare as they benefit from neither Cloudflare's dat…
No, thanks. Cloudflare acts ethically only until it suits them. It is the pre-exploitation phase to lure a customer. We are not fools here. The report at https://news.ycombinator.com/item?id=40481808 says it all. Secondly, considering Cloudflare would MITM all traffic, it would make a data good source for the NSA, thereby violating all user privacy.
This seems like a weak argument. Should we just take down anything widely accessed because it might be used by the NSA? What about AWS?
Earlier quoted context omitted.
I have asked this a few times and never gotten an answer beyond "One day they could turn evil." What is the reason Cloudflare is an extortionist firm? I am way more concerned about Amazon than Cloudflare.
"What is the reason Cloudflare is an extortionist firm?" Because they are a publicly traded company