251–260 of 261 posts
It's time for a lawyer letter. See the Computer Fraud and Abuse Act prosecution guidelines.[1] In general, the US Justice Department will not consider any access to open servers that's not clearly an attack to be "unauthorized access". But, "However, when authorizers later expressly revoke authorization—for example, through unambiguous written cease and desist communications that defendants receive and understand—the…
Earlier quoted context omitted.
I wonder how the author hasn't reached this conclusion. The official Amazon Crawler docs literally tell you how to distinguish between legit Amazonbots and malicious copycats via DNS lookup: https://developer.amazon.com/amazonbot
Why would someone copycat amazonbot?
It's been pounding one of my sites too. Here's the url it's trying to get to. I wonder if someone will ever figure it out and stop it. > /wp-content/uploads/2014/09/contact-us/referanslar/petrofac/wp-content/uploads/2014/09/products_and_services/products_and_services/catalogue/references/capabilities/about-company/wp-content/uploads/2014/09/wp-content/themes/domain/images/wp-content/uploads/2014/09/wp-content/uploads…
I bet that URL (or a predecessor) resolves to a 404 page that has a broken link on it. technically your problem.
It's time for a lawyer letter. See the Computer Fraud and Abuse Act prosecution guidelines.[1] In general, the US Justice Department will not consider any access to open servers that's not clearly an attack to be "unauthorized access". But, "However, when authorizers later expressly revoke authorization—for example, through unambiguous written cease and desist communications that defendants receive and understand—the…
Whats a process server in this context?
Earlier quoted context omitted.
HN isnt a monolith
Tell that to the moderation team.
There is one official mod who steps in occasionally. He is not the one flagging stories or comments. If a big story becomes unflagged, that is his doing more often than not
Earlier quoted context omitted.
Robots.txt is completely irrelevant. TOU/TOS are also irrelevant unless you restrict access to only those who have agreed to terms.
"By accessing this Site, you acknowledge that you have read, understand, and agree to abide by the terms described herein." https://www.imperva.com/legal/website-terms-of-use/ > Many, many, many hits for this or similar language: https://duckduckgo.com/?q=%22By+accessing+this+Site%2C+you+a... > Mind: just because it's written doesn't mean it's enforceable , but to argue that what you've just denied isn't a widely-use…
Earlier quoted context omitted.
"By accessing this Site, you acknowledge that you have read, understand, and agree to abide by the terms described herein." https://www.imperva.com/legal/website-terms-of-use/ > Many, many, many hits for this or similar language: https://duckduckgo.com/?q=%22By+accessing+this+Site%2C+you+a... > Mind: just because it's written doesn't mean it's enforceable , but to argue that what you've just denied isn't a widely-use…
Legal precedent in the US supports what I've stated.
Earlier quoted context omitted.
IA actually has technical and moral reasons to ignore robots.txt. Namely, they want to circumvent this stuff because their goal is to archive EVERYTHING.
Isn’t this a weak argument? OpenAI could also say their goal is to learn everything, feed it to AI, advance humanity etc etc.
Unless we start chopping these tech companies down there's not much hope for the public internet. They now have an incentive to crawl anything they can and have vastly more resources than even most governments. Most resources I need to host in a way that's internet facing are behind keyauth and I'm not sure I see a way around doing that for at least a while
That is an unworkable solution considering most of the people here are employed by these companies.