Live data from Hacker News

The 100k whys of AI

lcamtuf.substack.com

51–60 of 111 posts

Re: The 100k whys of AI

#51
post #34
post #2

A nice illustration of the homogeneity of LLM responses. Another way to describe this effect would be… If you ask humans to write 1,000 books, you're asking 1,000 different humans with different experiences and different skills and different moods (etc.) to write those books. But if you ask LLMs to write 1,000 books, you're probably only talking to 3 or 5 different models, tops. And they've all trained on the same or…

prompts will give very different results. this is where you do the work.

[dead]

Re: The 100k whys of AI

#52
The whole point of the thesis is that because the cover image are very similar, therefore LLMs are bad at writing text?

I think it's that today's LLMs have access to poor/generic image generation models and people find it easier to ask ChatGPT or NanoBanana to make a cover instead of fine tuning a small SD model for the purpose.

Re: The 100k whys of AI

#54
post #3

When you generate one or two blog posts with LLM they look pretty good. And you will be impressed with that one clever bit it adds that you didn't even ask for. But then you generate 50 of them and they all converge into the same pattern. It's hard to prove that an article is AI generated but they are instantly recognizable. An aside, I usually take my written blog posts through a pass on Notebooklm to generate a pod…

> they all converge AI is regression to the mean. Much like Socialism. Om an acute basis, AI can be just as helpful as that safety net. As a chronic matter, "it's not excellence--it's mediocrity".

And capitalism as seen in the USA is regression to the bottom of the cesspool?

Re: The 100k whys of AI

#55
post #2

A nice illustration of the homogeneity of LLM responses. Another way to describe this effect would be… If you ask humans to write 1,000 books, you're asking 1,000 different humans with different experiences and different skills and different moods (etc.) to write those books. But if you ask LLMs to write 1,000 books, you're probably only talking to 3 or 5 different models, tops. And they've all trained on the same or…

I don't think the comparison to humans works. It is as if you expect that we can easily train many different LLMs to solve the originality problem, but that is far from guaranteed.

Re: The 100k whys of AI

#56

Earlier quoted context omitted.

That whole thing would get you 1000 variants of existing art. But if you asked a thousand different designers to do a cover for the same book...

> 1000 variants of existing art. This is very naive. I can almost guarantee that some combinations of 20 * 50 features will hit on something that has never been written before in that specific combination . And if that's still not enough, increase the number of features. Add more randomness, add more steering, add random steering in random chapters, change it up, and so on.

I'm an art director. Finding a sequence that hasn't been hit in that specific combination is not sufficient to justify paying someone $150 an hour to go be creative.

Re: The 100k whys of AI

#57

Earlier quoted context omitted.

That whole thing would get you 1000 variants of existing art. But if you asked a thousand different designers to do a cover for the same book...

> 1000 variants of existing art. This is very naive. I can almost guarantee that some combinations of 20 * 50 features will hit on something that has never been written before in that specific combination . And if that's still not enough, increase the number of features. Add more randomness, add more steering, add random steering in random chapters, change it up, and so on.

> Add more randomness, add more steering, add random steering in random chapters, change it up, and so on.

That doesn't work for AI models. The whole training process depends on the basic principle that if you take the average of 100, in this case book cover designs, that the average is less like randomness than any individual cover you've used to make your average.

So the output will, by necessity, be closer to the average.

The human learning algorithm is much, much more data efficient than models. A absolute top human expert will have read/seen/heard/talked/... about 160 million "tokens" (that's about 2000 books). Frankly, the nerve inputs of all experiences of an entire human life, from baby to rewriting relativity theory, are only a couple dozen gigabytes.

Qwen 3.6 27B has been trained (as in seen ~10 to ~50 times) 8 trillion tokens, or to put it another way: for every second you will have spent "gathering life experiences" (ie. your whole life) on your deathbed Qwen 3.6 27B has spend about 50.000 seconds learning. And really that figure should be multiplied by the 10 or 50 training iterations.

Add another 3 or so orders of magnitude and you've got ChatGPT. By this measure, the human brains outperforms ridiculously overspecced ML models (because that's what ChatGPT and the like are) in efficiency a factor of by 5 million or more. This is the reason humans are still faster than ML models.

As for human training iterations: we can be simple: it's 1. In fact, it's impossible to make it even 2. Of course, when it comes to human performance: we are a better but not fundamentally different version of genetic algorithms. Do most humans perform? The honest answer is no. 1 in 1000, and that's very generous, improves SOTA. You absolutely need the 1000 failures though, as anyone whose tried a PhD (or even just design a large program) knows.

So we are very far away from allowing AI models to do what humans can do: take one example and produce, from one example, a better output. And there will always be much more variation in that approach. But ... most human attempts to do something are total crap. Most AI attempts to do something will succeed, but they'll be comparatively be bland, tasteless, "without soul", ...

And this is ignoring the problem that AI also has a massive limitation (that can't be solved, no matter how many nvidia cards you have) in that it trains against historical data. And counterfactuals don't work. What would have happened had Shakespeare decided Macbeth's wife was a force for good? Would the king still get murdered? Would it still be a great story? You can't work with counterfactuals.

Re: The 100k whys of AI

#58
What is worse, IMO, is that these GenAI books have found their way into physical stores. You know, the few that are still left.

I've found AI slop at many big box stores (think Walmart, Target, etc. and all their equivalents around the world) - which I suspect are "industry plants", meaning that the publishing house will have someone internally generate books like these, and sell them as physical copies around the thousands of stores I mentioned.

It is the equivalent of record labels pushing their own in-house GenAI artists.

Re: The 100k whys of AI

#59

What is worse, IMO, is that these GenAI books have found their way into physical stores. You know, the few that are still left. I've found AI slop at many big box stores (think Walmart, Target, etc. and all their equivalents around the world) - which I suspect are "industry plants", meaning that the publishing house will have someone internally generate books like these, and sell them as physical copies around the th…

There is a new “mural” (a 10m tall graffiti painting if you will) that is -obviously, but not ironically- an orange-toned AI designed picture, probably commmissioned bu the company the house belongs to ; makes my eyes bleed everytime

Re: The 100k whys of AI

#60

What is worse, IMO, is that these GenAI books have found their way into physical stores. You know, the few that are still left. I've found AI slop at many big box stores (think Walmart, Target, etc. and all their equivalents around the world) - which I suspect are "industry plants", meaning that the publishing house will have someone internally generate books like these, and sell them as physical copies around the th…

I'm sure they are banking on the possibility that the average book hoarder cannot tell the difference.
Post reply on HN