Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I prompted ChatGPT, Claude, Perplexity, and Gemini and watched my Nginx logs

surfacedby.com

Post reply on HN
Print view View original

21–28 of 28 posts

Re: I prompted ChatGPT, Claude, Perplexity, and Gemini and watched my Nginx logs

#21

Post by stronglikedan »

lambda wrote: post #8

Gah, the writing on this is so painful to read, it feels like this was most likely written by an LLM. The writing style is so unclear, it's hard to figure out one of the key points: it mentions that Gemini doesn't use a distinct user-agent for its grounding. It doesn't mention whether it actually hit the endpoint during the test, though it kind of implies that with "Silence from Google is not evidence of no fetch." U…

I haven't seen an LLM write this poorly yet (at least not passed off as good writing). This seems more like a person that used AI to organize things, but then didn't want it to seem like it was written by AI, so they rewrote it themselves. I think the problem here is just a genuinely unskilled author, and likely not a native English speaker judging by some of the awkward phrasing.

Re: I prompted ChatGPT, Claude, Perplexity, and Gemini and watched my Nginx logs

#23

Post by zenoprax »

startages wrote: post #14
lambda wrote: post #8

Gah, the writing on this is so painful to read, it feels like this was most likely written by an LLM. The writing style is so unclear, it's hard to figure out one of the key points: it mentions that Gemini doesn't use a distinct user-agent for its grounding. It doesn't mention whether it actually hit the endpoint during the test, though it kind of implies that with "Silence from Google is not evidence of no fetch." U…

I did use AI to organize my ideas but I didn't think it was that bad, I'll modify and make it easier to read. Anyway, in my test I saw zero requests from any Google UA after multiple Gemini and AI mode prompts that should have triggered grounding, so the working interpretation is that Gemini served from its own index/cache rather than doing a live provider-side fetch. The original phrasing was fuzzier than it should…

If this weren't on HN I wouldn't have given this more than a few seconds of reading before switching away. Some examples of phrasing that triggers me:

> attributing hits was a grep, not a guess > values below are copied from the probe’s log file, not paraphrased > a User-agent: Claude-User disallow is the live control > Only Claude-User is the user-initiated retrieval signal

I could go on and on but I won't. Phrasing aside, the text is too structured with many sections and subsections when the intent was clearly more narrative. "I was curious about X and did Y and I am going to tell you about it."

Signals that suggest a human who cares would be: use of the first-person; demonstrated curiosity, humility, and uncertainty; inline hyperlinks; and any kind of personality or opinion.

"Idiolect" is both subtle and distinct: the choice of vocabulary, grammar, phrasing and colloquial metaphors will vary in kind and frequency for everyone like an intellectual signature. You can sometimes tell if someone has been reading too much of a particular author recently just because of the way the author's choice of vocabulary bleeds into their own speech patterns. Sometimes it's a permanent influence.

I wonder if reading so much LLM stuff lately has affected my idiolect and that I write (or worse, think) more machine-like than before...

Re: I prompted ChatGPT, Claude, Perplexity, and Gemini and watched my Nginx logs

#24

Post by worik »

startages wrote: post #14
lambda wrote: post #8

Gah, the writing on this is so painful to read, it feels like this was most likely written by an LLM. The writing style is so unclear, it's hard to figure out one of the key points: it mentions that Gemini doesn't use a distinct user-agent for its grounding. It doesn't mention whether it actually hit the endpoint during the test, though it kind of implies that with "Silence from Google is not evidence of no fetch." U…

I did use AI to organize my ideas but I didn't think it was that bad, I'll modify and make it easier to read. Anyway, in my test I saw zero requests from any Google UA after multiple Gemini and AI mode prompts that should have triggered grounding, so the working interpretation is that Gemini served from its own index/cache rather than doing a live provider-side fetch. The original phrasing was fuzzier than it should…

[flagged]

Re: I prompted ChatGPT, Claude, Perplexity, and Gemini and watched my Nginx logs

#26

Post by reincoder »

I used the same methodology to observe AI crawlers. This is not an investigative blog but is rather designed to address our (IPinfo) customers who are asking us to identify IP addresses as "AI Agents" or, more accurately, "AI Crawlers".

https://community.ipinfo.io/t/can-we-detect-ai-agents-we-can...

Most AI crawlers self-identify with a UA. However, Grok uses resproxies and sends a high volume of simultaneous requests. Even though we can detect resproxies, it is not possible to map these resproxy IPs to grok.

I still could not figure out why I saw legitimate Googlebot IPs when I requested Perplexity to review the website. I verified those Googlebot IPs using both using UA and the listed IP address ranges published by Google.

Re: I prompted ChatGPT, Claude, Perplexity, and Gemini and watched my Nginx logs

#27

Post by ffsm8 »

zenoprax wrote: post #23
startages wrote: post #14

Earlier quoted context omitted.

I did use AI to organize my ideas but I didn't think it was that bad, I'll modify and make it easier to read. Anyway, in my test I saw zero requests from any Google UA after multiple Gemini and AI mode prompts that should have triggered grounding, so the working interpretation is that Gemini served from its own index/cache rather than doing a live provider-side fetch. The original phrasing was fuzzier than it should…

If this weren't on HN I wouldn't have given this more than a few seconds of reading before switching away. Some examples of phrasing that triggers me: > attributing hits was a grep, not a guess > values below are copied from the probe’s log file, not paraphrased > a User-agent: Claude-User disallow is the live control > Only Claude-User is the user-initiated retrieval signal I could go on and on but I won't. Phrasing…

> I wonder if reading so much LLM stuff lately has affected my idiolect and that I write (or worse, think) more machine-like than before...

Totally of topic ofc, but I always get triggered by the claim that llms are "machine-like". I'm aware it's a total pet peeve and a lil irrational, but "machine-like" would imply to me that it's thinking like a machine, which in turn implies machine intelligence - which in turn implies they're doing something which they aren't.

I'm not trying to undersell their capabilities. Used well they're able to do a lot of things. But the way they achieve it is by mimicking human dialogue and rhetoric processes to facilitate this process. That's in my opinion anything but machine intelligence. I struggle finding an applicable word for it though

Re: I prompted ChatGPT, Claude, Perplexity, and Gemini and watched my Nginx logs

#28

Post by zenoprax »

ffsm8 wrote: post #27
zenoprax wrote: post #23

Earlier quoted context omitted.

If this weren't on HN I wouldn't have given this more than a few seconds of reading before switching away. Some examples of phrasing that triggers me: > attributing hits was a grep, not a guess > values below are copied from the probe’s log file, not paraphrased > a User-agent: Claude-User disallow is the live control > Only Claude-User is the user-initiated retrieval signal I could go on and on but I won't. Phrasing…

> I wonder if reading so much LLM stuff lately has affected my idiolect and that I write (or worse, think) more machine-like than before... Totally of topic ofc, but I always get triggered by the claim that llms are "machine-like". I'm aware it's a total pet peeve and a lil irrational, but "machine-like" would imply to me that it's thinking like a machine , which in turn implies machine intelligence - which in turn i…

I didn't see your reply until now but "AI" is correctly describing the phenomenon. Most definitions for "artifice" converge around the idea of deception or insincerity.

The term "machine learning" also distinguishes itself from the organic process by authentic intelligence.

In other words, inferring "machine intelligence" is less correct than "artificial intelligence". By definition LLMs are machines pretending to think and they do it well enough to have a writing style.

Post reply on HN