71–80 of 84 posts
If you've ever tried scraping Facebook you'd know that it's nearly impossible to do so reliably. They have a formidable anti-scrape strategy. Instagram is currently ridiculously easy to scrape - but I doubt it's going to be that way for long.
Earlier quoted context omitted.
Legal rephrasing: Is there a reason that as a user of a user-agent like Firefox you would be in a legally different position than as a user of this user-agent app?
Yes, because your intent is to circumvent the provided mechanisms of the Instagram API and you’re doing it with a program instead of just browsing around the site with the intent of using it like a normal user who isn’t scraping. “But the program did the exact same thing that a human could do!” is a really good, pedantic CS argument, but it’s a terrible legal argument.
What is HN’s opinion on the legality of these types of scrapers? Instagram’s robots.txt disallows this kind of scraping, same for their ToS. Legal precedents have been mixed - the recent LinkedIn vs HiQ case is a good signal, but it’s still in appeals court.
Long story short I don't care about the "legality" of it. If it's publicly posted it's fair game. I don't care about tos or copyrights either. Everything is on the table.
Even made an issue on the repo when I ran into an issue setting it up on AWS and the maintainer was fast to respond.
Used it to make a bot to scrap soup special menus from a sandwich place near my work: http://blog.matthewbrunelle.com/projects/2018/05/07/Soup-Bot...
Earlier quoted context omitted.
That's why it is important to root out corruption from law making, nowadays called "lobbying" - this should be illegal. If you expose an endpoint to the public you can't restrict who or what can consume it. You can do throttling on your side but that's it. Otherwise this is just racism towards machines.
How do you outlaw lobbying?
If you've ever tried scraping Facebook you'd know that it's nearly impossible to do so reliably. They have a formidable anti-scrape strategy. Instagram is currently ridiculously easy to scrape - but I doubt it's going to be that way for long.
re.compile('(?:#)([A-Za-z0-9_](?:(?:[A-Za-z0-9_]|(?:\.(?!\.))){0,28}(?:[A-Za-z0-9_]))?)')
Earlier quoted context omitted.
I am comparing denial of right s on the basis of characteristic that subject can't change.
This is so myopic that I’m going to start a “sh%t HN says” Tumblr and make it the first post.
Yes it's an awful comment but it's one person, and it's been downvoted and flagged to death, which also says something about what "HN says".
What is this regex looking for?: re.compile('(?:#)([A-Za-z0-9_](?:(?:[A-Za-z0-9_]|(?:\.(?!\.))){0,28}(?:[A-Za-z0-9_]))?)')
Well, this is just a Python web scraper, and Instagram does in fact attempt to detect and prevent/rate-limit this kind of scraping. They rely very heavily on the source IP to help them determine when to cut you off.