Notably, in programming this is actually a desirable feature for most problems. Even human programmers are taught to produce predictable and obvious code whenever possible. I wonder is ultimately this is an artifact of optimizing the models for code, that they become less creative.
The 100k whys of AI
11–20 of 111 posts
Re: The 100k whys of AI
#12Ignore me
> This is a fuzzy signal, so you shouldn’t fire your intern when they say “it’s not this — it’s that”.
The author literally points to that tell in the article.
In a weird twist, I wonder if you’re an LLM?
Re: The 100k whys of AI
#13Ignore me
You're either sarcastic or you missed all of: a) this being in italics, b) this being in quotes, c) this being the only LLM pattern in the post, d) this being quoted explicitly as an LLM marker. Good job in both cases, I guess.
Re: The 100k whys of AI
#14Have you seen Egyptian paintings or Hollywood movies?
Everything is slop if you make enough of it and squint hard enough.
The point with AI is if and how to steer it to produce something that is interesting and unique for you, not another bland cookie cutter blockbuster or lame summer song.
Re: The 100k whys of AI
#15A nice illustration of the homogeneity of LLM responses. Another way to describe this effect would be… If you ask humans to write 1,000 books, you're asking 1,000 different humans with different experiences and different skills and different moods (etc.) to write those books. But if you ask LLMs to write 1,000 books, you're probably only talking to 3 or 5 different models, tops. And they've all trained on the same or…
that discounts, how much the other context, ie, the system, prompt, and any sort of other context submitted to the model that can affect the output. If you ask a model as a patient for medical advice versus as a doctor, you will get different output from the same model.
Re: The 100k whys of AI
#16A nice illustration of the homogeneity of LLM responses. Another way to describe this effect would be… If you ask humans to write 1,000 books, you're asking 1,000 different humans with different experiences and different skills and different moods (etc.) to write those books. But if you ask LLMs to write 1,000 books, you're probably only talking to 3 or 5 different models, tops. And they've all trained on the same or…
> you're asking 1,000 different humans with different experiences and different skills and different moods Simply, if you ask an LLM, you're asking always to the same mind, and always for the first time.
Also since those are lazy, you are also asking always in the same manner. How homogeneous were the prompts that generated those covers?
People are making cookies with cookie cutter number 5 and other people wonder how come they are all the same.
Re: The 100k whys of AI
#17When you generate one or two blog posts with LLM they look pretty good. And you will be impressed with that one clever bit it adds that you didn't even ask for. But then you generate 50 of them and they all converge into the same pattern. It's hard to prove that an article is AI generated but they are instantly recognizable. An aside, I usually take my written blog posts through a pass on Notebooklm to generate a pod…
I suspect there are new invariants emerging. We don’t know what they are and we will probably have to reach into the liberal arts to describe them but to me what you’re seeing is akin to the subatomic world exposing itself through diffraction patterns.
Re: The 100k whys of AI
#18I love the illustration of the same-ness of AI.
One question / quibble:
> if a hundred “authors” give their favorite AI tool a similar prompt
Do we really believe there are 100 different people generating those? When I saw the books, I assumed they were generated on demand to match the (to me unlikely) search terms.
I don’t think I’m invested enough to research this. Amazon slop is harder and harder to wade through. (Searches are very imprecise. Deliberate, I’m sure.)
Re: The 100k whys of AI
#19On HN many comments under many threads are about whether the submission was written by AI. You could say I have noticed a pattern in Hacker News comments!
In these comments there's a common pattern where some users argue that they do not agree that the submission was LLM written and they often focus on specific details to refute it (e.g em-dashes) and some users see the overall pattern clearly that it's totally obvious. For me it's a kind of smell, it's off putting and it's obvious. The article says to "trust your gut". But it's also something that comes with practice and time, it's not some innate thing. People may have better things to do than expend mental energy noticing patterns in a bunch of social media posts. The more I see it, the more I see it.
The take away I get is that it's okay to notice patterns and it's okay to not notice patterns. Remember that other people may be noticing patterns and associations in things that you might miss. Be charitable.
Far more interesting questions are:
1) If you cant see the patterns of LLM writing, does the idea that the thing you liked was written by LLM worry you?
2) If you can see the patterns clearly is the fact that it's LLM written worry you?
Because in our comments there's many who do not care that LLM's are writing content and theres many who do care. But are these correlated with those who can see the LLMs or who are blind to them?
Re: The 100k whys of AI
#20Notably, in programming this is actually a desirable feature for most problems. Even human programmers are taught to produce predictable and obvious code whenever possible. I wonder is ultimately this is an artifact of optimizing the models for code, that they become less creative.
Determinism is a desirable property for software, yes, and its lack thereof from LLM’s is a common complaint, but often a feature depending on who you ask. There is an element of randomness, “hotness” that is central to who LLM work but the pattern we see here manifesting reveals the deterministic processes below, but I don’t think you could rely on this technology to be deterministic, if that’s what you wanted.