I noticed sec.gov is blocking all requests from AWS since the 4 days ago. Has this been the same for you? How are you able to bypass this issue? Currently I can scrape sec.gov only from local computer. My ideal solution is that I can have a program that runs on some remote servers
Sec.gov is blocking all requests from AWS?
1–3 of 3 posts
Re: Sec.gov is blocking all requests from AWS?
#2You could use a proxy run on another provider, Linode, DigitalOcean, etc. Or you could scrape sec.report
What are you using SEC sites for, just EDGAR or something else?
Re: Sec.gov is blocking all requests from AWS?
#3Add a user agent to the request