61–70 of 84 posts
If you've ever tried scraping Facebook you'd know that it's nearly impossible to do so reliably. They have a formidable anti-scrape strategy. Instagram is currently ridiculously easy to scrape - but I doubt it's going to be that way for long.
Earlier quoted context omitted.
> Am I the only one... Yes, because you’re comparing racism and slavery to not being able to use your computer to scrape data from websites. Edit: don’t ask for opinions if you can’t handle some of them.
I am comparing denial of right s on the basis of characteristic that subject can't change.
What is HN’s opinion on the legality of these types of scrapers? Instagram’s robots.txt disallows this kind of scraping, same for their ToS. Legal precedents have been mixed - the recent LinkedIn vs HiQ case is a good signal, but it’s still in appeals court.
It sucks that the same companies that are trying to have scraping be ilegal, do massive scraping on their end, many times deceiving the users whose data they are scraping. FB and LinkedIn both scrape contacts (and who knows what else), from their users email accounts (and surely lots of other darker sources as well). Pretty much all of Google's search engine's content is scraped from other websites.
I agree with you, but technically that argument is true
And as for Google, in theory a robots.txt would block their scraper.
If you've ever tried scraping Facebook you'd know that it's nearly impossible to do so reliably. They have a formidable anti-scrape strategy. Instagram is currently ridiculously easy to scrape - but I doubt it's going to be that way for long.
what do they do that makes it so hard?
This scraper was written to get images and create a dataset for ML models for personal project while studying Machine Learning and Artificial Intelligence.
If you've ever tried scraping Facebook you'd know that it's nearly impossible to do so reliably. They have a formidable anti-scrape strategy. Instagram is currently ridiculously easy to scrape - but I doubt it's going to be that way for long.
what do they do that makes it so hard?
Earlier quoted context omitted.
If you don't want me to scrape it, don't put it on the (public) internet. Simple as that.
I don't really get this attitude of because it can be done it's therefore ok. It's possible to kill people therefore it's ok. It's possible to loiter therefore it's okay. It's possible to play your music at all hours of the day at 120db therefore its ok. It's possible to drive your motorcycle into a grocery store therefore it's okay. The physical world has plenty of laws and/or common sense/customs about what behavio…
Earlier quoted context omitted.
what do they do that makes it so hard?
They randomize the tag ids and classes a lot of the time. At least that has been my experience with the Facebook messenger website.
Earlier quoted context omitted.
Legal rephrasing: Is there a reason that as a user of a user-agent like Firefox you would be in a legally different position than as a user of this user-agent app?
Yes, because your intent is to circumvent the provided mechanisms of the Instagram API and you’re doing it with a program instead of just browsing around the site with the intent of using it like a normal user who isn’t scraping. “But the program did the exact same thing that a human could do!” is a really good, pedantic CS argument, but it’s a terrible legal argument.
https://en.wikipedia.org/wiki/Browse_wrap#Summary
Remember, anyone can sue you for anything and it is then your burden to defend yourself if they are persistent and wealthy enough.
This scraper was written to get images and create a dataset for ML models for personal project while studying Machine Learning and Artificial Intelligence.
as somebody that has fielded numerous emails from friends asking me to remove tagged photos of them from flickr, i sort of wonder about the ethics of harvesting these sorts of images from instagram, a community whose norms sort of revolve around semi-public sharing of photos. I don't doubt that there's some rationale for harvesting the images from ig, but aside from thumbing your nose at their TOS, it feels like it's a greater violation of trust to harvest your friends and strangers photos for an ML project without their informed consent.
at the very least, it's worth considering pointing your app's gaze at a set of images licensed for any purpose whatsoever rather than ones that are explicitly licensed All Rights Reserved by their respective photographers.