Live data from Hacker News

The AI bullshit singularity

successfulsoftware.net

161–170 of 187 posts

Re: The AI bullshit singularity

#161

I think that people's believe in LLM is intelligence is closely tied to mistaken association between intelligence and the ability to speak. Parrots can speak, but they cannot reason. Moreover, evolutionary, birds learned to mimic the speech to fool other species. To fool in such way that other species would think that parrots are of the same specie. We, the humans, are smart enough to recognise that even though parro…

Hinton thinks LLMs can "understand" because without understanding it's impossible to predict the next work as effectively as GPT-4. The only way to do that is to understand the meaning in the text. He also says he's given GPT-4 a novel reasoning problem he invented and it successfully answered, which wouldn't be possible without a level of understanding/intelligence (although I'm not sure how he ensured the problem w…

Well, my point was that the speech by itself is not a good criteria to estimate the intellect. This is not only applicable to machines but to humans as well. I assume that the primary evolutionary determined purpose of a speech function was convincing rather than the source of reasoning. Even though we use natural languages to broadcast the information, the languages usually overwhelmed with linguistics and psychological tricks to make an illusion of usefulness and novelty of this information. Even if the information was truly novel, it's usually hard or even impossible to find the roots. This is one of the reasons of why most people prefer to learn rather to invent. Broadcasting of information of inventions made by someone else is much simpler and usually more beneficial than researching something from scratch. And our natural language specifically designed for such forms of broadcastings.

In this sense if the ML developers would be focused on the inventions automatisation, I would expect that they would choose something more formalised than the natural language.

Anyway, whatever way and methods they chose I think the better external estimation criteria of intelligence should be the ability to make completely new things that clearly didn't exist before. Not just reasoning about existing one. After all, it's not a new thing that computers are able to deduce. Any programming language can do that better than any chat bot.

Re: The AI bullshit singularity

#162

Earlier quoted context omitted.

You're forgetting that we have mechanisms in place to curate curation and farm attention toward the crap, not toward quality. It will be harder and harder to break the surface tension of this gooey gelatin wrapper we've placed over creative activity.

People will pay money for curation towards quality, probably quite a bit if the rest of the landscape is 99.99% noise. They won't pay much money for curation "toward the crap". So I'm not nearly as pessimistic.

But that will be reputation-based, and names can be sold. Look at the brand-holding conglomerates we have today that don't make any of the original product, they just license the name out to anyone, regardless of the quality of the finished good. From mattresses to magazines, we keep seeing this. Why wouldn't we see this in curation sites?

Re: The AI bullshit singularity

#163

Earlier quoted context omitted.

People will pay money for curation towards quality, probably quite a bit if the rest of the landscape is 99.99% noise. They won't pay much money for curation "toward the crap". So I'm not nearly as pessimistic.

But that will be reputation-based, and names can be sold. Look at the brand-holding conglomerates we have today that don't make any of the original product, they just license the name out to anyone, regardless of the quality of the finished good. From mattresses to magazines, we keep seeing this. Why wouldn't we see this in curation sites?

Sure, there will always be some fraction of dupes and double games being played, but how does that change the main point?

It's very unlikely for 100% of all intelligent curators to become dishonest either so it doesn't seem like a serious roadblock.

Re: The AI bullshit singularity

#164
post #2

I broadly agree with that. Repeated training with self-generated data is the technological equivalent of incest and can lead to nothing good.

Training on self generated data is not necessarily such a problem, see how successful alphazero/muzero is when it is only trained on self play. The key is that you need some kind of external indicator that tells you which generated examples are good and which are bad. In the case of alphazero you get that by simulating games and seeing who wins, in the case of LLMs you will be only taking the generations that are 'su…

Let's indeed hope that it will still be humans doing most of the voting on HN.

Re: The AI bullshit singularity

#165
post #80

Earlier quoted context omitted.

I think it’s clear that LLMs cannot be the end state of this technology, and we will need systems that can reason and develop hypotheses and test them internally. These systems may benefit from more curated datasets (such as those collected before the bullshit wave began) along with real world interaction data from YouTube and robotics. Such systems could eventually be used to rank web pages for their bullshit level,…

> It’s just really clear that a giant text averaging machine can only go so far It's not really a text averaging machine, it's a pattern matching machine. Right now the "depth" of the patterns it can match can only go so far, but in a few years with more advances in chips and memory the depth is going to increase and the patterns it can match will fan out accordingly.

it is a statistical model. If everyone is saying X and is wrong, and one guy says Y and is right, the llm will bit out X. Because that is the most probable thing in the dataset. It literally is a text averaging machine

Re: The AI bullshit singularity

#166

Succinctly stated and something that resonates strongly with me. In the last internet revolution (web search), results started high quality because the inputs were high quality - bloggers and others just wanted to document and share knowledge. But over time, many interests (largely commercial) figured out how to game the system with SEO, and quality of search results has decreased as search's incentive structure led…

I guess some of it might depend on how good the AI-generated content gets, and also how good the AI gets at detecting AI-generated content. If the AI was good at detecting it, it wouldn't matter if the AI-generated content sucked, yes? Even a low probability of detection would help. Let's say our algorithm is 50% likely to detect AI junk. That means that half the junk data won't make it in to model. Even 20% would pr…

ai detectors can be used to train ais to avoid triggering them. It's a race that cannot be won. ai detection can't, and already doesn't, work.

Re: The AI bullshit singularity

#167

As AI advances, there will be AI and algorithms that will check LLMs and their output, sort of LLMs' verifier. It is foolish to think that LLMs won't improve to the point where there are almost no or very low hallucinations and false information. We are just at the beginning of LLMs and generative AI.

"The queen is ________"

The model predicts a 99% chance that the next word is "alive" and a 1% chance that it's "dead".

The llm calculates probabilities. How does it actually choose the word? It throws a weighted die. It literally chooses one at random (albeit on a custom probability distribution).

So tell me, what how can you eliminate hallucinations from something that is literally designed to pick stuff at random?

Hallucinations will never be removed from these types of llms. Hallucinations are fundamental to how they work. In the sense that even the "good" outputs are hallucinations picked out at random from a probability distribution.

Any company that says they can control hallucinations, in any way, is flat out lying.

Re: The AI bullshit singularity

#168
post #78

Earlier quoted context omitted.

It's getting really tiresome to see tech bros talk about biology as if they know what they're talking about.

> It's getting really tiresome to see tech bros talk about biology as if they know what they're talking about. My undergrad was in biology, "bro". I was cloning luciferace into plants using agrobacterium-mediated transfection over a decade before this week's news of transgenic petunias. I was planning to do a PhD in computational metabolomics, but life took a different turn: a couple of Google engineers saw the laser…

If you studied biology then you ought to know better.

Re: The AI bullshit singularity

#169
post #87

Earlier quoted context omitted.

I think it’s clear that LLMs cannot be the end state of this technology, and we will need systems that can reason and develop hypotheses and test them internally. These systems may benefit from more curated datasets (such as those collected before the bullshit wave began) along with real world interaction data from YouTube and robotics. Such systems could eventually be used to rank web pages for their bullshit level,…

Not just real world data from videos. AI models need feedback from many sources: humans, code execution, web search, simulations, games, robotics, math verification, or from actual experiments in the real world. All of these are environments that can take the output of a model and do some processing and return feedback. The model can learn and search for solutions, creating its own training data as a RL agent. Since…

But bullshit hallucinogenic output and fakery at colossal scales doesn't just pollute the pool of static information available on the web. It also warps the minds of the humans you're relying on to verify reality on the next training loop. Not only that, but we can also see the emergent breed of humans who believe that writing is similar to arithmetic - an unnecessary skill that can be handed off to a calculator. Or that making a movie shouldn't require knowing anything besides asking for what you want to see. How is someone like that - someone who wants to rely on bots - going to tell a bot what is or isn't true? How can they even have a pre-pollution baseline understanding of reality?

I just finished rereading Do Androids Dream for the first time in 20 years or so, and was astonished at how similar his andys really are to LLMs in the polluted / destroyed reality there. How confusing and corrupting they are to organic life. PKD describes the androif brains as neural networks with thousands of layered pathways and trillions of weighted parameters, and it's as if he was able to accurately conceive of what linguistic and "emotional" strengths and weaknesses those constructs would actually have, decades before LLMs existed. And there's this one amazing line where Deckard calls them "Life thieves". What else should we call what Sam Altman and others are building - but theft of human ingenuity, creativity, and basic reason for living, and the utter annihilation and suppression of people like this 16 year old kid who dare to hope they can contribute something more original in life than being a servant of a tech company building this shit, or a tiktok influencer who writes prompts?

What should that kid hope: That their work becomes noticeable enough to be immediately stolen and their name turned into a prompt?

Life thieves.

Just as a further aside, I had dinner tonight with a friend who's a fairly famous animator in the commercial realm, and I brought up this post. He's just sure the kid's screwed and the genie is out and creative is basically over. He's turning to building wooden clocks.

But his reaction, and the reactions I see here every time this comes up, remind me of something else. They remind me of how people react when someone is robbed. Everyone has some reason why it's bad but not that bad, it was inevitable, it'll be okay, etc. Or they go around wondering what they're going to do now. Or they paper it over with optimism. Surprisingly few people get robbed and are willing to realize they were robbed, and become wildly pissed off about it. Most people have some sort of flight reaction, as evidenced by e.g. the promotion of "prompt writing" or learning to use a paid API to do what was formerly your own creative job that now spits your own work back at you.

I say sue the shit out of all these content thieves.

Re: The AI bullshit singularity

#170
post #5

The same goes for image generation models, AI art already has a tendency to veer into the same clichés and those are only going to get reinforced if newer models are trained on newer scrapes which now include the million hyper-derivative AI images being uploaded to places like DeviantArt, Twitter and Pixiv every day. Those vendors who got in early have a moat in the form of untainted scrapes, but they'll eventually n…

Art is much less vulnerable. Art is very heavily tagged, accurately described and filtered. Many galleries don't accept AI art at all, which means to sneak in the AI needs to be pretty much perfect. Also, there's an enthusiastic scene of LoRAs where the makers work with small enough datasets to do manual curation.

>> which means to sneak in the AI needs to be pretty much perfect

That's status quo. Because right now artificial art is easy to distinguish form "natural art". Besides that I consider this - no offense! - as some kind of "arrogance". Galleries accept what they think what art is. Why can't I, the art consumer, decide for myself, what art is?

Post reply on HN