Like everything else on the web, LLMs are going to eventually be ruined by marketing teams trying to get them to say "Pepsi" instead of "Coke."
Long live local LLMs!
Blocking LLMs from your website cuts you off from next-generation search
41–50 of 82 posts
Re: Blocking LLMs from your website cuts you off from next-generation search
#42This post gets the reason why people are cutting off LLMs exactly backwards and consequently completely fails to address the core issue. The whole reason people are blocking LLMs is precisely that they believe it kills the flow of readers to your content. The LLMs present your ideas and content, maybe with super-tiny attribution that nobody notices or uses [1], maybe with no attribution at all, and you get nothing. P…
Re: Blocking LLMs from your website cuts you off from next-generation search
#43Good, because this “next generation search” doesn’t cite sources, invents falsehoods, steals content, and doesn’t direct traffic to the site in question, which was the whole point of search engines in the first place . The fact LLM companies constantly keep getting dinged for ignoring every barrier we throw up to stop their scraping short of something like Anubis shows what their real goal is: theft, monopolization,…
Re: Blocking LLMs from your website cuts you off from next-generation search
#44Good, because this “next generation search” doesn’t cite sources, invents falsehoods, steals content, and doesn’t direct traffic to the site in question, which was the whole point of search engines in the first place . The fact LLM companies constantly keep getting dinged for ignoring every barrier we throw up to stop their scraping short of something like Anubis shows what their real goal is: theft, monopolization,…
Re: Blocking LLMs from your website cuts you off from next-generation search
#45Good, because this “next generation search” doesn’t cite sources, invents falsehoods, steals content, and doesn’t direct traffic to the site in question, which was the whole point of search engines in the first place . The fact LLM companies constantly keep getting dinged for ignoring every barrier we throw up to stop their scraping short of something like Anubis shows what their real goal is: theft, monopolization,…
If your website is included in next gen search and the user asks for a source, then your website as a source will be included.
https://www.pewresearch.org/short-reads/2025/07/22/google-us...
Re: Blocking LLMs from your website cuts you off from next-generation search
#46This post gets the reason why people are cutting off LLMs exactly backwards and consequently completely fails to address the core issue. The whole reason people are blocking LLMs is precisely that they believe it kills the flow of readers to your content. The LLMs present your ideas and content, maybe with super-tiny attribution that nobody notices or uses [1], maybe with no attribution at all, and you get nothing. P…
Fair, if your content is your product, but I’m more than happy for every LLM on the planet to summarize my page and hype the virtues of my product to its user.
Re: Blocking LLMs from your website cuts you off from next-generation search
#47Re: Blocking LLMs from your website cuts you off from next-generation search
#48I agree with the sentiment. I remember Gwern in an interview remarking something to the effect that if you make your writing and thoughts invisible to LLMs, then your thoughts are going to be invisible to the future, as LLMs are here to stay.
Re: Blocking LLMs from your website cuts you off from next-generation search
#49Earlier quoted context omitted.
Fair, if your content is your product, but I’m more than happy for every LLM on the planet to summarize my page and hype the virtues of my product to its user.
Why do tech bros assume that every site is selling a product? There are blogs, personal web sites, communities, and open-source projects out there.