Why we're taking legal action against SerpApi's unlawful scraping

Post by bluGill » Fri, Dec 19, 2025, 8:54 PM UTC

pawelduda wrote: post #33

bluGill wrote: post #7

Earlier quoted context omitted.

Google publishes how to control their bot - with robots.txt. They then obey those instructions. Google also takes some effort to not use all your bandwidth. Google isn't perfect, but they are at least making a "good faith" effort to be nice and this does count in court. Overall most will agree that in general what google does to allow people to find their website is worth the things that google is doing. You can of c…

But their robots are enabled by default. So it is a form of unsolicited scraping. If I spam millions of email addresses without asking for permission but provide a link to opt-out form, am I the good guy?

Post by [deleted] » Fri, Dec 19, 2025, 9:44 PM UTC

bitpush wrote: post #41

AstroBen wrote: post #31

Earlier quoted context omitted.

Because they've forced everyone to allow them. They're the internet traffic mafia. Block them and you disappear from the internet They abuse this power to scrape your work, summarize it and cut you out as much as possible. Pure value extraction of others' work without equal return. Now intensified with AI But yeah, you're right. They're not deceptive

> Because they've forced everyone to allow them. nobody is forcing anyone. This is the same argument that people said about google search. Nobody is forcing anyone to use google search, google chrome, or even allow googlebot for scraping. Thousands of poeple have switched over to chatgpt, brave/firefox .. Your argument sounds like "I dont like Apple's practices, and I'm forced to buy iPhones. No buddy, if you dont li…

Post by bluGill » Fri, Dec 19, 2025, 9:58 PM UTC

pawelduda wrote: post #45

bluGill wrote: post #42

Earlier quoted context omitted.

At this point everyone knows about robots.txt, so if you didn't opt-out that is your own fault. Opting out of everyone at once is easy, and you get fine grained control if you want it. Also most people would agree they are fine with being indexed in general. That is different from email spam where people don't want it.

Looking at SerpApi clients, looks like most companies would agree they are fine with scraping Google. That is different from having your website content stolen and summarized by AI on Google search, which people don't want.

Post by [deleted] » Fri, Dec 19, 2025, 11:20 PM UTC

bluGill wrote: post #46

pawelduda wrote: post #45

Earlier quoted context omitted.

Looking at SerpApi clients, looks like most companies would agree they are fine with scraping Google. That is different from having your website content stolen and summarized by AI on Google search, which people don't want.

The claim is SerApi is not honoring robots.txt, and they are getting far more data from google/more often than needed for an index operation. Or at least that is the best I can make out of the claim in court from the article - I have not read the actual complaint. People are generally fine with indexing operations so long as you don't use too much bandwidth. Using AI to summarize content is still and open question -…

Post by skybrian » Sat, Dec 20, 2025, 12:59 AM UTC

Nextgrid wrote: post #29

skybrian wrote: post #8

Earlier quoted context omitted.

As far as I know, Google respects robots.txt and doesn't obfuscate their crawlers, so you can easily block them if you want. It seems like an important distinction?

Google can afford to respect robots.txt because it has a monopoly on search and nobody would consider actually blocking them in said robots.txt anyway. SerpApi doesn't have that privilege.