Earlier quoted context omitted.
I wish they'd limit it to just stopping credential stuffing. Here's my scenario: My electricity provider publishes the month's electricity rates on the first of the month, I want to scrape these so that I can update the prices in Home Assistant. This is a very simple task, and it's something that Home Assistant can do with a little configuration. Unfortunately this worked exactly once, after that it started serving u…
Check if your electricity provider offers Green Button Download. https://www.energy.gov/data/green-button
Re: Avoiding bot detection: How to scrape the web without getting blocked?
#311Seconded. My company offers this and it's a great resource.