I prompted ChatGPT, Claude, Perplexity, and Gemini and watched my Nginx logs
1–10 of 28 posts
Re: I prompted ChatGPT, Claude, Perplexity, and Gemini and watched my Nginx logs
#2Re: I prompted ChatGPT, Claude, Perplexity, and Gemini and watched my Nginx logs
#3Re: I prompted ChatGPT, Claude, Perplexity, and Gemini and watched my Nginx logs
#4Re: I prompted ChatGPT, Claude, Perplexity, and Gemini and watched my Nginx logs
#5Re: I prompted ChatGPT, Claude, Perplexity, and Gemini and watched my Nginx logs
#6The content is interesting, but it's delivered in an article that smells like slop.
Re: I prompted ChatGPT, Claude, Perplexity, and Gemini and watched my Nginx logs
#7Re: I prompted ChatGPT, Claude, Perplexity, and Gemini and watched my Nginx logs
#8The writing style is so unclear, it's hard to figure out one of the key points: it mentions that Gemini doesn't use a distinct user-agent for its grounding. It doesn't mention whether it actually hit the endpoint during the test, though it kind of implies that with "Silence from Google is not evidence of no fetch." Uh, if there are no requests coming in live, that means no fetch, it's using a cache of your site.
It makes a difference whether it fetches a page live, or whether it's using a cached copy from a previous crawl; that tells you something about how up-to-date answers are going to be from people asking questions about your website from Gemini. But I guess the LLM writing this article just wanted to make things sound punchy an impressive, not actually communicate useful information.
Anyhow, LLM marketing spam from an LLM marketing spam company. Bleh.
Re: I prompted ChatGPT, Claude, Perplexity, and Gemini and watched my Nginx logs
#9There are multiple ways these tools access your site and only one of them is “using it for training”. Others are webfetch from chat sessions, “deep research” agents, etc. And those will have different traffic patterns. They aren’t crawlers, they are clumsy, ham handed AI agents doing their humans bidding.
Both can give a site the hug of death. Both can be badly coded. But there is much different intent behind the two and I feel it is important to acknowledge the difference.
Re: I prompted ChatGPT, Claude, Perplexity, and Gemini and watched my Nginx logs
#10The IPs listed in the output are from reserved ranges as well, like they were intentionally obfuscated (but this was not shared with the reader).
It’s the kind of obfuscation that AI would do (using esoteric bogon ranges as well)