Recursion into madness • Hacker News Board

Re: Recursion into madness

#31

Post by Sharlin » Fri, Sep 11, 2026, 2:23 PM UTC

blargey wrote: post #14

Speculated to be responsible for the "AI restaurant-menu gore" that made the rounds on social media. Restaurants AI-generate the entire menu sheet from whole cloth, and while it looks "good" at first with modern image models, repeated edits to the menu text/layout transform the passable food "photos" into trypophobia-inducing horrors. (And somehow they don't notice or care) Notably, the recent ChatGPT Images 2.5 anno…

With local models it's always been easy to edit only a part of an image (inpainting), either by painting a mask or automatically using a small segmentation model. It's actually pretty strange that commercial models have resorted to redrawing the entire image even in cases where the requested change is localized.

Re: Recursion into madness

#32

Post by MarkusQ » Fri, Sep 11, 2026, 2:55 PM UTC

layer8 wrote: post #7

MarkusQ wrote: post #6

It would have been interesting to try something closer to the movie poster example; say have the AI rewrite the passage with the room toggling between the initial light-mode decor and a dark-mode decor, then back, repeatedly. Or even change the woman's name to "Mrs. Smythe" and back.

As the article explains: “[…] In other words, if you instruct a model to change a single word in a paragraph of text, it can almost always handle the task with no collateral damage.“

Right, but if you ask it to make a systematic change (the room decor for light to dark and back) or something with subtle implications that would require other changes to fit in?

Changing the title in the movie example is this sort of change; the "change a single word" isn't analogous.

Re: Recursion into madness

#33

Post by rcxdude » Fri, Sep 11, 2026, 3:28 PM UTC

GPerson wrote: post #12

Does the same effect happen in a society which rewrites all of its knowledge over and over using AI?

I would argue it's a risk of any process that loops around itself without grounding in reality periodically.

Re: Recursion into madness

#34

Post by sophrosyne42 » Fri, Sep 11, 2026, 4:28 PM UTC

xiaoyu2006 wrote: post #23

brzezmac wrote: post #22

that's why I argue the LLMs are just a bunch of glorified lossy compression engines ;)

Be cautious of that claim. Lossy compression may exactly hold intelligence.

Probably not, since our intellect can produce lossless compression too

Re: Recursion into madness

#35

Post by tencentshill » Fri, Sep 11, 2026, 6:43 PM UTC

GPerson wrote: post #12

Does the same effect happen in a society which rewrites all of its knowledge over and over using AI?

It's why any "synthetic training data" startup is doomed to fail.