341–350 of 555 posts
The author just seems to have a hate for AI and a less than practical understanding of what happens when you put things on the internet.
Earlier quoted context omitted.
What is a "visit"? TFA demonstrates that they got a hit on their site, that's how they got the logs. Is it necessary to load the JavaScript for it to count as a visit? What if I access the site with noscript? Or is it only a visit if I see all your recommended content? I usually block those recommendations so that I don't get distracted from the article I actually came to read—is my visit a less legitimate visit than…
A visit is a human reader. At the very least they get exposed to your website name. Notice your product/service if you get lucky. Become a customer at a later visit. We are talking about cutting the first step off so that everything which may come afterwards is cut off as well.
[0] Which, incidentally, is entirely normal. robots.txt is for the web crawler that indexes, not for ad hoc requests.
OpenAI scraped aggressively for years. Why should others put themselves behind an artificial moat? If you want to block access to a site, stop relying on arbitrary opt-in voluntary things like user agent or robots.txt. Make your site authenticated only, that’s literally the only answer here.
... And what would you do to find out if an image or piece of text (that you didn't create) has been plagiarised by an AI model? You search for it on a search engine, of course!
Earlier quoted context omitted.
I agree with that, but I also think that they should at least identify themselves instead of using a generic user agent.
I’d rather share less information than more to any site I visit. Why does a user want to share that info?
How does that impact users at all?
A lot of comments here are confusing the two use cases for crawling: training and summarization. Perplexity's utility as an answer engine is RAG (retrieval augmented generation). In response to your question, they search the web, crawl relevant URLs and summarize them. They do include citations in their response to the user, but in practice no one clicks through on the tiny (1), (2) links to go to the source. So if y…
I would even argue that the whole conversation about AI is a distraction here. Imagine if, instead of using an LLM, Perplexity actually assigned a human agent to your query who'd do the same thing that the model does: write the search queries based on your high-level question, read through the pages that come up, and condense it all into a summary with references to the original sources. That would, of course, be a lot more expensive, but the output would be the same, and so would be the consequences: the person who asked the original high-level question does not get exposed to all the content that had to be waded through to answer it. Is that unethical? If not, then why does replacing the human agent with an AI in this scenario becomes unethical?
And if the answer is "scale", that gets uncomfortably close to saying that it's okay for the rich but not for the plebs.
Earlier quoted context omitted.
> It’s retrieving the content then manipulating it. Perplexity isn’t a web browser. So a browser with an ad-blocker that's removing / manipulating elements on the page isn't a browser? What about reader mode?
How a user views a page isn't the same as a startup scraping the internet wholesale for financial gain.
Respecting robots.txt is something their training crawler should do, and I see no reason why their user agent (i.e. user asks it to retrieve a web page, it does) should, as it isn't a crawler (doesn't walk the graph). As to "lying" about their user agents - this is 2024, the "User-Agent" header is considered a combination bug and privacy issue, all major browsers lie about being a browser that was popular many years…
They might be "lying" because of all sorts of reasons, but a specific version of Chrome on a specific OS still sends a unique user agent string.
Earlier quoted context omitted.
What will happen if: Website owners decide to stop publishing because it’s not rewarded by a real human visit anymore? Then perplexity and the like won’t have new information to train their models on and no sites to answer the questions. I think there is a real content dilemma here at work. The incentives of Google and website owners were more or less aligned. This is not the case with perplexity.
> I think there is a real content dilemma here at work It's not really a dilemma. This is exactly what copyright serves to protect authors from. Perplexity copied the content, and in doing so directly competes with the original work, destroying it's market value and driving the original author out of business. Literally what copyright was invented to prevent. It's the exact same situation as journalists going after G…
Earlier quoted context omitted.
I'm not sure what you mean exactly. If Perplexity is actually doing something with your article in-band (e.g. downloading it, processing it, and present that processed article to the user) then they're just breaking the law. I've never used that tool (and don't plan to) so I don't know. If they just embed the content in an iframe or something then there's no issue (but then there's no need or point in scraping). If t…
Sure it is, but which of the many small websites are going to be able to fight them legally? Most companies would go broke before getting a ruling. Reality is, the law doesn't matter if you're big enough. As long as they're not stealing content from the big ones, they're going to be fine.
Earlier quoted context omitted.
How would an LLM training on your writing reduce your reward? I guess if you're doing it for a living sure, but most content I consume online is created without incentive (social media, blogs, stack overflow). I write a fair amount and have been for a few years. I like to play with ideas. If an llm learned from my writing and it helped me propagate my ideas, I'd be happy. I lose on social status imaginary internet po…
I think a concern for people who contribute on Stack Overflow is that an LLM will pollute the water with so many subtly wrong answers that the collective work of answering questions accurately will be overwhelmed by a tsunami of inaccurate LLM-generated answers, more than an army of humans can keep up with checking and debugging (or debunking).
For what it's worth, the Stack Exchange terms of service do prohibit AI generated content. I'm not sure how they actually enforce that, and in practice as the LLMs improve it's going to be almost impossible to reliably detect.