Live data from Hacker News

Ask HN: Is scraping webpages for research purposes considered legal

news.ycombinator.com

1–4 of 4 posts

Re: Ask HN: Is scraping webpages for research purposes considered legal

#3
post #2

Depends on a country. And in practice on how much money the people behind the website can throw at the lawyers when annoyed. For the US you may want to read about https://en.wikipedia.org/wiki/Aaron_Swartz

Also depends on the ToS of each page, and their robots.txt should allow or disallow crawling