Perplexity AI is lying about their user agent
1–10 of 555 posts
Re: Perplexity AI is lying about their user agent
#2VC/Big tech company is stealing data until it damage their PR and sometimes they never stops, sadly nothing new in current tech world.
Re: Perplexity AI is lying about their user agent
#3I don’t think we should lump together “AI company scraping a website to train their base model” and “AI tool retrieving a web page because I asked it to”. At least, those should be two different user agents so you have the option to block one and not the other.
Re: Perplexity AI is lying about their user agent
#4Our bot traffic is up 10-fold since LLM Cambrian explosion.
Re: Perplexity AI is lying about their user agent
#5Read this article if you want to know Perplexity’s idea of taking other people’s content and thinking they can get away with it,
https://stackdiary.com/perplexity-has-a-plagiarism-problem/
The CEO said that they have some “rough edges” to figure out, but their entire product is built on stealing people’s content. And apparently[0] they want to start paying big publishers to make all that noise go away.
[0]: https://www.semafor.com/article/06/12/2024/perplexity-was-pl...
Re: Perplexity AI is lying about their user agent
#6The only way out seems to be using obscene captcha.
Re: Perplexity AI is lying about their user agent
#7How about a trap URL in the Robots.txt file that triggers a 24 hour IP ban if you access it.
If you don't want anyone innocent caught in the crossfire, you could make the triggering URL customized to their IP address.
Re: Perplexity AI is lying about their user agent
#8Our bot traffic is up 10-fold since LLM Cambrian explosion.
Cambrian explosion implies that there’s a huge variety of different creatures out there, but I suspect those bots are all just wrappers around OpenAI/anthropic models.
This is more like the rise of Cyanobacteria as a single early dominant lifeform
Re: Perplexity AI is lying about their user agent
#9I don’t think we should lump together “AI company scraping a website to train their base model” and “AI tool retrieving a web page because I asked it to”. At least, those should be two different user agents so you have the option to block one and not the other.
Is it actually retrieving the page on the fly though? How do you know this? Even if it were - it’s not supposed to be able to.
Re: Perplexity AI is lying about their user agent
#10What incentive does anybody have to be honest about their user agent?