Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why we're taking legal action against SerpApi's unlawful scraping

blog.google

Post reply on HN
Print view View original

61–70 of 113 posts

Re: Why we're taking legal action against SerpApi's unlawful scraping

#61

Post by hn_acker »

bluGill wrote: post #54
khelavastr wrote: post #52

Earlier quoted context omitted.

Who says robots.txt is legally binding? Where's the Sherman Antitrust analysis?I'm more confused than before.

The courts say. With this as a long standing tradition they are likely to agree.

> The courts say.

Do you have an example of a court saying that violating robots.txt violates an existing law?

In Ziff Davis v. OpenAI [1], the District Court for the Southern District of New York found that violating robots.txt does not violate DMCA section 1201(a) (formally 17 U.S. Code § 1201(a), which prohibits circumvention of technological protection measures of copyrighted content [2]).

It's my understanding that robots.txt started as a socially-enforced rule and that it remains legally voluntary.

[1] https://blog.ericgoldman.org/archives/2025/12/are-robots-txt...

[2] https://www.law.cornell.edu/uscode/text/17/1201

Re: Why we're taking legal action against SerpApi's unlawful scraping

#62

Post by ddtaylor »

Google really doesn't have a leg to stand on here. They scrape the Internet. They replace content against the wishes of users multiple different times, such as with AMP. Their entire business model recently has been to provide you answers they learned from scraping your website and now they want to sue other people who are doing the same.

Data wants to be free. They knew that once.

EDIT: Also to be clear I am not saying they can't win legally. I'm sure they can do legal games and could shop around until they were successful. They are in the wrong conceptually.

Re: Why we're taking legal action against SerpApi's unlawful scraping

#63

Post by polishdude20 »

"Google follows industry-standard crawling protocols, and honors websites’ directives over crawling of their content."

Is that true with how they trained Gemini? Doesn't everyone with a foundational model scrape the web relentlessly without regard for robots.txt?

Re: Why we're taking legal action against SerpApi's unlawful scraping

#64

Post by jonatron »

polishdude20 wrote: post #63

"Google follows industry-standard crawling protocols, and honors websites’ directives over crawling of their content." Is that true with how they trained Gemini? Doesn't everyone with a foundational model scrape the web relentlessly without regard for robots.txt?

They're in a unique position where many people allow googlebot but try to block most other bots

Re: Why we're taking legal action against SerpApi's unlawful scraping

#68

Post by tenuousemphasis »

jonatron wrote: post #64
polishdude20 wrote: post #63

"Google follows industry-standard crawling protocols, and honors websites’ directives over crawling of their content." Is that true with how they trained Gemini? Doesn't everyone with a foundational model scrape the web relentlessly without regard for robots.txt?

They're in a unique position where many people allow googlebot but try to block most other bots

Allow for the purpose of indexing, not training models.

Like if you give a friend a key to your house so they can check on your plants when you're out of town but they throw a rager and trash the place.

Re: Why we're taking legal action against SerpApi's unlawful scraping

#69

Post by maplethorpe »

polishdude20 wrote: post #63

"Google follows industry-standard crawling protocols, and honors websites’ directives over crawling of their content." Is that true with how they trained Gemini? Doesn't everyone with a foundational model scrape the web relentlessly without regard for robots.txt?

[flagged]

Re: Why we're taking legal action against SerpApi's unlawful scraping

#70

Post by eddythompson80 »

ddtaylor wrote: post #62

Google really doesn't have a leg to stand on here. They scrape the Internet. They replace content against the wishes of users multiple different times, such as with AMP. Their entire business model recently has been to provide you answers they learned from scraping your website and now they want to sue other people who are doing the same. Data wants to be free. They knew that once. EDIT: Also to be clear I am not say…

Eh, and in 20 if SerpApi or whatever the fuck becomes the next google, they’ll have a blog post titled “Why we’re taking legal action against BlemFlamApi data collection”.

The biggest joke was all the “hackers” 25 years ago shouting “Don’t be evil like Oracle, Microsoft, Apple or Adobe and charge for your software, be good like Google and just put like a banner ad or something and give it away for free”

Post reply on HN