111–120 of 212 posts
Earlier quoted context omitted.
Is this strictly legal? For example, in the scenario where a "misconfigured" bot of a large evil corporation get's taken down and, due to layers of ass covering, they think it's your fault and it cost them a lot of money. Do they have a legal case that could fly in eastern district of Texas?
Texas, America?
Earlier quoted context omitted.
Yes, it requires the client to try and extract the archive; https://en.wikipedia.org/wiki/Zip_bomb is the generic description.
What archive? The idea was to use Transfer-Encoding: gzip, which means the compression is a transparent part of the HTTP request which the client HTTP library will automatically try to extract.
Earlier quoted context omitted.
Interesting. What does number 5 do? Also, how do gzip bombs works, does it automatically extract to the 20gb or the bot has to initiate the extraction?
Most HTTP libraries would happily extract the result for you. [citation needed]
Python package requests
Whatever is the default these days in C#
Honestly, I have never used a modern HTTP client library that does not automatically decompress.
I guess libCurl might be a case where you need to add an option to force decompress.
First off, make a website defend mode that can be triggered to serve different content. Then, do the following: 1. Add a robots.txt, make it look like it's wordpress (Disallow: /wp-admin etc) 2. If any client requests /wp-admin, flag their IP ASN as bot. 3. If a client is a bot, send it a gzip bomb (100kB size, unpacked around 20GB), use Transfer-Encoding: gzip and Content-Encoding: gzip to also punish malicious web…
I spent 2 minutes of my life shooting cookies with a laser. I also spent close to a quarter of a minute poking a cookie.
Earlier quoted context omitted.
If you ban an IP or even an ASN, there could be (many) thousands sharing that same identifier. Some kid will unknowingly run some free game that does some lightweight scraping in the background as monetization and you ban the whole ISP?
How would that be a false positive? The kid might not be malicious, but they absolutely are running a bot, even if unknowingly. If anything, calling attention to it could help people notice, and therefore clean up such things.
I would be add some fun with colors - modulate them. Not much, I think would be enough to change color temperature to warm or cold, but same color.
Content of modulation could be some sort of fun pictures, may be videos for most active bots.
So if bot put converted colors to one place (convert image), would seen ghosts.
Could add some Easter eggs for hackers - also possible conversion channel.
Earlier quoted context omitted.
How would that be a false positive? The kid might not be malicious, but they absolutely are running a bot, even if unknowingly. If anything, calling attention to it could help people notice, and therefore clean up such things.
The kid isn't. But everyone else using their ISP that your ASN-based block also blocks is a false positive. An ASN block easily has a granularity of "10% of an entire large country". And nobody is going to take your site blocking e.g. all Comcast users as "oh, we should investigate which Comcast user made some slightly suspicious requests, thanks for telling us".
So is that such a bad thing? If OP is going to use this to provide data about bots, blocking mass amounts of the internet could actually be a terrific example of how many people are at least tangentially connected to bots.
First off, make a website defend mode that can be triggered to serve different content. Then, do the following: 1. Add a robots.txt, make it look like it's wordpress (Disallow: /wp-admin etc) 2. If any client requests /wp-admin, flag their IP ASN as bot. 3. If a client is a bot, send it a gzip bomb (100kB size, unpacked around 20GB), use Transfer-Encoding: gzip and Content-Encoding: gzip to also punish malicious web…
Genuinely interested in your thinking: superficially looking, your anti-bot ideas are a bit contradictory to your Stealth browser, which enables bots. Why did you choose to make your browser useful for bot activity? [1] https://github.com/tholian-network/stealth
Being able to use the web anonymously is a value that should be held up against moral values. Malicious scraping of websites should be, too. Both ideas can simultaneously true, they don't have to be mutually exclusive. I also support initiatives like the web archive which I consider "good behavior" of web scraping.
If a person asks me for the dataset of my website and they don't have a competitive business that they run, I'm even happy to open source the dataset generation part. Contributors get more rights, abusers get less rights. That's how it should be in my opinion.
I don't see them as contradictory.