It will force more robust filtering and analysis
that will allow to determine "quality value" of
any information piece and grade its novelty/uniqueness
for classification and subsequent training,
which will use only high-quality(book-grade)
and unique/novel content(excluding copypasted SEO spam).
The AI bullshit singularity
11–20 of 187 posts
Re: The AI bullshit singularity
#12There is quite a lot of bullshit in human generated content on the internet.
Re: The AI bullshit singularity
#13The same goes for image generation models, AI art already has a tendency to veer into the same clichés and those are only going to get reinforced if newer models are trained on newer scrapes which now include the million hyper-derivative AI images being uploaded to places like DeviantArt, Twitter and Pixiv every day. Those vendors who got in early have a moat in the form of untainted scrapes, but they'll eventually n…
Are 'scrapes' really the only acceptable source?
Re: The AI bullshit singularity
#14I broadly agree with that. Repeated training with self-generated data is the technological equivalent of incest and can lead to nothing good.
Putting washing machine parts in the laundry, irradiating pre-war steel, etc.
Re: The AI bullshit singularity
#15I always found the idea of infinitely self improving AI to be suspect. Let’s say we have a super smart AI with intelligence 1, and it uses all that to improve itself by 0.5. Then that new 1.5 uses itself to improve by 0.25. Then 0.125, etc etc. obviously it’s always increasing, but it’s not going to have the runaway effect people think.
If it improves at a faster rate than humanity, it pulls ahead even if the absolute speed is slow. That's what people are really more worried about, not instant omniscience.
Re: The AI bullshit singularity
#16I broadly agree with that. Repeated training with self-generated data is the technological equivalent of incest and can lead to nothing good.
Only let baby see its own scribbles and scat.
Future world leader advisor.
Ahem, I hope not.
Re: The AI bullshit singularity
#17When the ELIZA chatbot came out (1964), some users claimed it was intelligent and sentient. Then, they were shown the diagram that describes its behavior. Even after that, they kept insisting it was sentient, etc. Some people just want to believe bullshit.
When you interact with an LLM (or Eliza) your brain is doing a lot of heavy lifting, without you even realizing it. I think we tend to infer more intelligence than is there, in the same way we see faces in clouds.
Re: The AI bullshit singularity
#18The same goes for image generation models, AI art already has a tendency to veer into the same clichés and those are only going to get reinforced if newer models are trained on newer scrapes which now include the million hyper-derivative AI images being uploaded to places like DeviantArt, Twitter and Pixiv every day. Those vendors who got in early have a moat in the form of untainted scrapes, but they'll eventually n…
This is such a wrong take.
Even if the data is 100% synthetic, you can still hill climb to new mountains.
If you don't believe me, look at evolution.
It doesn't matter if we no longer have 100% human art as input. This is the worst these systems will ever look and feel, and they're only going to improve.
I'd be willing to do a longbets on this one.
Re: The AI bullshit singularity
#19The web is well on its way to destroying itself, LLMs are only going to accelerate the process.
Re: The AI bullshit singularity
#20Succinctly stated and something that resonates strongly with me.
In the last internet revolution (web search), results started high quality because the inputs were high quality - bloggers and others just wanted to document and share knowledge. But over time, many interests (largely commercial) figured out how to game the system with SEO, and quality of search results has decreased as search's incentive structure led to lower quality data being indexed.
We're at the start of the LLM revolution now - models are trained on high quality inputs (which may be as rare as "low-background steel" in the future). But the models allow the mass production of lower quality outputs with errors and hallucinations; once those get fed back into new models, are we doomed to decreasing effectiveness of LLMs, just as we've seen with search? Will there be LLMO (LLM optimization) to try to get your commercial interests reflected in the next generation models?
I think we've got a few golden years of high quality LLMs before that negative feedback loop really starts to hurt like it did in search.