Viewing profile — gigaArpit

Robots.txt is a joke, use Cloudflare's Block AI Bots feature if you are using Cloudflare.

Following this reasoning, a 200 response would also be appropriate, as AI bots wouldn't flag anything as suspicious.

The idea behind introducing a new HTTP status code for blocking AI traffic should be to provide a clearer signal to both users and automated systems about the nature of the respons…

Each code has a defined meaning, which helps clients and servers understand the nature of the response.