Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

If AI scaling is to be shut down, let it be for a coherent reason

scottaaronson.blog

Post reply on HN
Print view View original

151–160 of 475 posts

Re: If AI scaling is to be shut down, let it be for a coherent reason

#151

Post by bombcar »

mrob wrote: post #137
bombcar wrote: post #84

Earlier quoted context omitted.

Work out the details on exactly how said end-of-the-world would occur. Note that we already have “AI to end cities” - they’re just sitting, turned off, waiting for the code and button press in silos and submarines throughout the world.

The danger is from something vastly more intelligent than humans, and with a mindset that's incomprehensibly alien. No human is capable of working out the details. That doesn't mean the risk doesn't exist. Failure to understand chemistry does not make ants immune to insecticides. The only thing we can assume about a super-intelligence is that it will be highly capable of achieving its goals. There is no reason to ass…

So humans will build God and so we all best get religion, and fast?

Re: If AI scaling is to be shut down, let it be for a coherent reason

#152

Post by kami8845 »

nl wrote: post #59
detrites wrote: post #43

Earlier quoted context omitted.

A great many would likely join you, making this entire fiasco a time-wasting distraction, at best, and grave risk at worst. The technologies will continue to be developed, moratorium or not. A moratorium only enables the hidden to get there first. The risks need to be discussed and understood, along with the benefits, and publicly. That's the only sensible way forward. Denying that the technology is here already and…

Of course they would. It's just ridiculous. If people are genuinely concerned about lack of access to the OpenAI models then work at training open ones! OpenAI has a maybe 6 month lead and that's nothing. Plus it's much easier being the follower when you know what is possible. (To be clear, I know at least a few of the projects already working on this. I just want to make it clear that is the intellectually honest ap…

I think their lead might be a bit bigger than that. ChatGPT 3.5 was released 4 months ago and I still haven't seen another LLM come close to it.

Re: If AI scaling is to be shut down, let it be for a coherent reason

#153

Post by l5ymep »

gspencley wrote: post #94
skocznymroczny wrote: post #54

Am I the only one who's not very concerned about ChatGPT and "AI" in general? I am young, but still lived through several hype phases. I remember when 3D TVs were going to be mainstream and 2D was considered legacy. I remember when PC was to die soon and to be replaced by smartphones. I remember when VR was to become as common accessory as a game controller. It's 2023, and I still don't have automated self driving ca…

You don't even have to look at other "fads" like 3D televisions and VR to be skeptical of the claim that recent advancements in ML* represent a major technological revolution that will change society like the Internet or the Printing Press did. Just look at ML's own history. A few years ago we had the "deepfake" scare. People were terrified that now videos are going to surface of politicians saying and doing things t…

Other fads have the disadvantage of being easily identifiable and avoidable. But AI chips away at what it means to be human. Now imagine if every comment in this thread is not a real person, but made by a LLM. Get any Truman show vibes?

Re: If AI scaling is to be shut down, let it be for a coherent reason

#154

Post by dandellion »

mrob wrote: post #141
dandellion wrote: post #92

Earlier quoted context omitted.

I don't think it's so strange to expect the reaction to be proportionate to the threat they see? An open letter seems several orders of magnitude shorter to anything that would be effective if it really is a threat to the species. I think of examples of ways people react when there are very high stakes: resigning in protest, going on a hunger strike, demonstrations/raids, setting themselves on fire, etc. But there's…

If a terrorist bombs one data center, security increases at all the other data centers. Bombing all data centers (and chip fabs, so they can't be rebuilt) simultaneously requires state-level resources.

Going down that line of thought not even a state could realistically bomb the data centers of all states, it's kind of pointless. But I wasn't really arguing that they need to destroy all datasources, but rather that raiding a datacenter would be more appropriate response to the threats the claiming exist. They wouldn't even need to succeed in vandalising one, they'd just have to try.

Re: If AI scaling is to be shut down, let it be for a coherent reason

#155

Post by lukev »

rhn_mk1 wrote: post #133
lukev wrote: post #125

Earlier quoted context omitted.

This is exactly the kind of mysticism I'm talking about. In fact we know precisely how LLMs work. The fact that parts of human linguistic concept-space can be encoded in a high dimensional space of floating point numbers, and that a particular sequence of matrix multiplications can leverage that to perform basic reasoning tasks is surprising and interesting and useful . But we know everything about how how it is trai…

We know how they work, that is true. We don't know why they work, because if we could, then we could extrapolate what happens when you throw more compute at them, and no one would have been surprised about the capabilities of GPT-N+1. Also no one would have been caught with their pants down by seeing people jailbreak their models. To illustrate it in a different way: on a mechanistic level, we know how animal brains…

I don't need examples. It's simply how they work. This is why they hallucinate.

A LLM is fundamentally a mathematical function (albeit a very complex one, with billions of terms (a.k.a parameters or weights)). The function does one thing and one thing only: it takes a sequence of tokens as input (the context), and it emits the next token(word)[1].

This is a stateless process: it has no "memory" and the model parameters are immutable; they are not changed during the generation process.

In order to generate longer sequences of text, you call the function multiple times, each time appending the previously generated token to the input sequence. The output of the function is 100% dependent on the input.

Therefore, the only "internal state" a model has is the input sequence, which is human-readable sequence of tokens. It can't "hallucinate", it can't "lie", and it can't "tell the truth", it can only emit tokens one at a time. It can't have a hidden "intent" without emitting those tokens, it can't "believe" something different than what it emits.

[1] Actually a set of probabilities for the next token, and one is selected at random based on the "heat" generating setting, but this is irrelevant for the high-level view.

Re: If AI scaling is to be shut down, let it be for a coherent reason

#156

Post by [deleted] »

nradov wrote: post #122
GistNoesis wrote: post #102

Here is a coherent reason : Just to know where we are standing in our ability to control AI. Like an alcoholic saying he is in control and that he can stop whenever he wants, can try to go dry for a month (or 6) to show that he still somewhat is. If Covid exposed only one thing : it is humanity total failure to control global phenomenons for positive outcomes. It is becoming evident to everybody, and this open-letter…

There was never any realistic possibility of controlling a highly contagious respiratory virus, so COVID-19 didn't expose anything about humanity's ability to control global phenomenons. What it did expose was the tendency of many people to irrationally overreact to a relatively minor risk, mostly because the risk was novel .

[deleted]

Re: If AI scaling is to be shut down, let it be for a coherent reason

#157

Post by pwinnski »

skocznymroczny wrote: post #54

Am I the only one who's not very concerned about ChatGPT and "AI" in general? I am young, but still lived through several hype phases. I remember when 3D TVs were going to be mainstream and 2D was considered legacy. I remember when PC was to die soon and to be replaced by smartphones. I remember when VR was to become as common accessory as a game controller. It's 2023, and I still don't have automated self driving ca…

The thing about predicting doom is that you're wrong often--until one day you're not.

Most hype cycles involve people at the top making wild claims that fail to deliver. I don't know anyone outside the industry who ever thought 3D TVs were worth anything, and barely anyone who thought VR was worth anything. Google pitched AI making appointments, but that never made it off a stage. Hype? Only for some definition of hype.

Smartphones have changed the world, but it was primarily Apple who pushed the "post-PC" narrative, and that was to promote the iPad, one of their least successful product lines. (To be clear: it's still a HUGE business and success, but it didn't usher in the post-PC world Steve Jobs claimed it would.)

One you left out is cryptocurrency, and that's the only one I can think of where the hype came from more than just the people at the top, mostly because everyone down the chain thought they were also people at the top by virtue of buying in. Financial scams are always hype by their nature.

I'm older than some, younger than others, but in more than 30 years as a professional developer, I think this is as close to a "silver bullet" as I've ever seen. Like GUIs and IDEs, I think LLMs are tools that will make some things much easier, while making other things slightly harder, and will generally make developers as a class more productive.

There's no question that developers using a nice IDE to write high-level code on a large monitor are able to product more code more quickly than someone writing assembler on a dumb terminal, I hope. The shift from monochrome ASCII to GUIs helps, the shift from a text editor to an auto-completing, stub-writing IDE helps, and similarly, I think having an LLM offer up a first pass approximation for any given problem definition helps.

Concerned? I'm not concerned, I'm excited! This isn't marketing hype coming from someone on a stage, it's grassroots hype coming from nobodies like me who are finding it actually-helpful as a tool.

Re: If AI scaling is to be shut down, let it be for a coherent reason

#158

Post by Ygg2 »

jacquesm wrote: post #64
skocznymroczny wrote: post #54

Am I the only one who's not very concerned about ChatGPT and "AI" in general? I am young, but still lived through several hype phases. I remember when 3D TVs were going to be mainstream and 2D was considered legacy. I remember when PC was to die soon and to be replaced by smartphones. I remember when VR was to become as common accessory as a game controller. It's 2023, and I still don't have automated self driving ca…

> I am young, but still lived through several hype phases. My impression is that this is a tech revolution unlike any that has gone before allowing for absolutely massive concentration of power and wealth. The previous tech revolutions have shown the downsides of such concentration and I'm quite sure that the people that are currently in the driving seat of how, when and where this tech gets used and/or weaponized ar…

> Extrapolating from the last two years

Therein lies the error. People forget reality is finite and just because it improved now doesn't mean it will continue indefinitely.

  An exponential curve is just a sigmoid curve in disguise.
Most AI I've seen suffer from catastrophic errors in the tail end (famous example of two near identical cat pictures classified as cat and dog respectively).

Re: If AI scaling is to be shut down, let it be for a coherent reason

#159

Post by credit_guy »

To me the signators (is that the word?) of the letter are extremely naive.

The horse is out of the barn. Out of the farm, the county, and the state.

Yesterday Bloomberg released their own LLM. You can bet dollars to pennies that lots of other firms are working on their own LLM's. Are they going to stop because of a letter? Well, you can say, they will stop if the letter results in an act of Congress. First, the Congress will not be so silly as to impose a handicap on the US firms, knowing full well that Russian and Chinese firms will not respect the moratorium. But even if Congress were to consider this moratorium, you think all these firms furiously working on their proprietary LLM's will sit idle? That none of them ever heard the word "lobbying"? But even if they don't lobby by some miracle, and the law is passed, you think they will not find a loophole? For example, will Congress not allow for a Defense exemption? Can't you then train your LLM for some Defense purpose, and then use the weights for some other purposes?

If you pass a law to stop LLM's, the only thing you achieve is to increase the barriers of entry. It's like picking winners and losers using the criterion that you win if you are rich and lose if you are poor.

Re: If AI scaling is to be shut down, let it be for a coherent reason

#160

Post by famouswaffles »

lukev wrote: post #125
rhn_mk1 wrote: post #78

Earlier quoted context omitted.

> Because they can imagine a god-like super intelligent AI, it must be possible. It's worse. It may be possible, but we're not equipped to recognize the line as it's crossed. Combined with us making LLMs more and more capable despite not knowing why they work, this extrapolating of LLMs to gods is not insane.

This is exactly the kind of mysticism I'm talking about. In fact we know precisely how LLMs work. The fact that parts of human linguistic concept-space can be encoded in a high dimensional space of floating point numbers, and that a particular sequence of matrix multiplications can leverage that to perform basic reasoning tasks is surprising and interesting and useful . But we know everything about how how it is trai…

We don't know how they work lol. How they are trained is what we understand. Nobody knows what the models learn exactly during training and nobody sure as hell knows what those billions of neurons are doing at inference. Why just a few months ago, some researchers discovered the neuron that largely decides when "an" comes before a word in GPT-2. We understand very little about the inner workings of these models. And if you knew what you were talking about, you would know that.
Post reply on HN