Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

If AI scaling is to be shut down, let it be for a coherent reason

scottaaronson.blog

Post reply on HN
Print view View original

461–470 of 475 posts

Re: If AI scaling is to be shut down, let it be for a coherent reason

#461

Post by Nasrudith »

l5ymep wrote: post #153
gspencley wrote: post #94

Earlier quoted context omitted.

You don't even have to look at other "fads" like 3D televisions and VR to be skeptical of the claim that recent advancements in ML* represent a major technological revolution that will change society like the Internet or the Printing Press did. Just look at ML's own history. A few years ago we had the "deepfake" scare. People were terrified that now videos are going to surface of politicians saying and doing things t…

Other fads have the disadvantage of being easily identifiable and avoidable. But AI chips away at what it means to be human. Now imagine if every comment in this thread is not a real person, but made by a LLM. Get any Truman show vibes?

If AI can chip away at your meaning of being human I am afraid your definition and understanding before is in need of improvement. Destroying a faulty understanding should be celebrated, not feared.

Re: If AI scaling is to be shut down, let it be for a coherent reason

#462

Post by carapace »

kypro wrote: post #454
carapace wrote: post #424

Earlier quoted context omitted.

> I mean no offence (sic) None taken. > such an unthoughtful take No offense, but if you were 10 in 1999 when Iron Giant came out then I have been thinking about this for longer than you have been alive. > Of course there's a cost? Well, effectively zero as compared to the open-ended benefit of keeping pristine humanity around. Study ecology. There is no overlap between the ecological niches of non-biological sentien…

Well, I like this comment better, and for the record I have no problem with you being condescending, so please speak plainly as need be. > Well, effectively zero as compared to the open-ended benefit of keeping pristine humanity around. Could you expand? What is the benefit of keeping 9 billion resource hungry inferior life forms around and all of the biological matter needed to support them? > Study ecology. There i…

> Well, I like this comment better, and for the record I have no problem with you being condescending, so please speak plainly as need be.

Cheers!

> Could you expand? What is the benefit of keeping 9 billion resource hungry inferior life forms around and all of the biological matter needed to support them?

Sure, and thanks for asking. From Information Theory we have the result that the unpredictability of a message is a measure of its information content.

The entire rest of the known Universe has approximately thirty bits of information, as it can be completely described by a handful of equations. There is a vast amount of data, but almost no information, in the entire rest of the Universe.

In contrast, the living biosphere is... there's really no way to describe it in words, even mathematically it's kind of ridiculous to represent how much information the Earth generates.

To beings whose food is information, the biosphere is infinitely valuable, their attitude is akin to worship, the way plants feel about the sun.

> Depth isn't a very relevant metric here, it's about how the surface area is used.

The surface area of the Earth is irrelevant, they don't want to live here. They will live in space, where it's flat and cold and there's a constant energy and particle flux. (E.g. half of the Earth is eclipsed by the Earth at all times, eh?)

The entire Earth is approximately infinitesimal to creatures whose natural habitat extends to the heliopause.

Re: If AI scaling is to be shut down, let it be for a coherent reason

#464

Post by rolisz »

kilotaras wrote: post #460
rolisz wrote: post #445

Earlier quoted context omitted.

I think that the moment you scale up the hardware for intelligence (from the 20cm brain) to a datacenter scale (to several hundred meters) you start getting into some other kinds of issues, such as it's hard to get a consistent view across the whole "brain". Synchronizing things will take longer, limited by the speed of light: suddenly things on the far ends can communicate with a latency with a lower bound of 100ns.…

> But we do have super human chess engines, but those are made in a completely different way from GPT. Suppose GPT-X just dropped and it could both generate text AND play chess better than Magnus Carlssen. How would that change you position on it being unable to pass the human-level intelligence barrier?

That would make me rethink my position.

Re: If AI scaling is to be shut down, let it be for a coherent reason

#465

Post by famouswaffles »

lukev wrote: post #393
famouswaffles wrote: post #374

Earlier quoted context omitted.

It is all things considered pretty easy to set up GPT such that it runs on its own input forever while being able to interact with users/other systems. add an inner monologue/react and reflexion and you have a very powerful system. embody it with some physical locomotive machine and oh boy. no one has really put this all together yet but everything i've said has been done to some degree. The individual pieces are her…

I agree that LLMs could be one module in a future AGI system. I disagree that LLMs are good at simulation. They're good at prediction . They can only simulate to the degree that the thing they're simulating is present in their training data. Also, if you were trying to build an AGI, why would you NOT run it slowly at first so you could preserve and observe the logs? And if you wanted to build it to run full speed, wh…

human.exe, robot.exe, malignant agent.exe are all very much simulations that an llm would have no problem running.

>Also, if you were trying to build an AGI, why would you NOT run it slowly at first so you could preserve and observe the logs?

I'm telling you that is extremely easy to do all the things i've said. Some might be interested in doing what you say. Others might not. at any rate, to be effective this requires real time monitoring of thoughts and actions. That's not feasible forever. an LLMs state can change. There's no guarantee the friendly agent you observed today will be friendly tomorrow.

>And if you wanted to build it to run full speed, why would you not build other single-purpose dumber AIs to watch it in case its thought stream diverged from expected behavior?

This is already done with say Bing. Not even remotely robust enough.

Re: If AI scaling is to be shut down, let it be for a coherent reason

#466

Post by nl »

NumberWangMan wrote: post #236
nl wrote: post #187

Earlier quoted context omitted.

I think that's a subtly different thing. The OPs claim was more or less the paperclip maximizer problem. I contend that a super intelligence given a specific goal by humans would take the context of humans into account and avoid harm because that's the intelligent thing to do - by definition. The orthogonal thesis is about the separation of intelligence from goals. My attitude to that is that a AI might not actually…

Hmm, why would you say that avoiding harm is the intelligent thing to do, by definition?

Do you have a better definition?

Re: If AI scaling is to be shut down, let it be for a coherent reason

#467

Post by ChatGTP »

slowmovintarget wrote: post #338
bvaisvil wrote: post #240

Earlier quoted context omitted.

Who's gonna pay for GPT's in an economic collapse?

You're presupposing the collapse touches the rich. "A quart of wheat for a denarius, and three quarts of barley for a denarius; but do not harm the oil and the wine!"

So how does this situation actually look:

Bill Gates sitting on a yacht controlling a robot army while we're all starving ?

Re: If AI scaling is to be shut down, let it be for a coherent reason

#468

Post by ChatGTP »

barking_biscuit wrote: post #448
tome wrote: post #442

Earlier quoted context omitted.

> You do realize we've just at least doubled the amount of cognitive bandwidth on earth right? I don't realize that. What do you mean exactly?

LLMs can perform cognitive labor. Moreover they can increasingly perform it at a level that matches and/or surpasses humans. Humans can use them to augment their own capabilities, and execute faster on more difficult tasks with a higher rate of success. In addition, cognitive labor can be automated. When bandwidth increases, latency decreases. We're going to be able to get significantly higher throughput than we were…

Nope, it's now another voice, a colleague who's work you constantly have to check over because you can't trust it enough to just say, "go ahead, I know you're trust worthy".

This is already happening. ??

Re: If AI scaling is to be shut down, let it be for a coherent reason

#469

Post by bamboozled »

bigtex88 wrote: post #258
computerex wrote: post #14

I read the times article, and I also watched Lex Fridman's talk with Eliezer Yudkowsky. Frankly I don't think he is qualified, and I don't understand why anyone is even giving him any credence. His argument is literally: > A mis-aligned super AGI will result in the death of humanity I ask, how . He is making reductive logical leaps that don't make sense. It's FUD.

A mis-aligned super AGI will treat the Earth and everything on it as a playground of atoms and material. Why wouldn't it? What do children do when they see a sandbox? Do they care what happens to any ants or other bugs that might live in it? There does not need to be a "how", as you put it. The logic is "Maybe we should tread carefully when creating an intelligence that is magnitudes beyond our own". The logic is "Ma…

If such a thing as a super intelligence existed, why would it operate on matter in the physical world?

Animals do physical world modifications because we're biological and need shelter etc.

A super intelligence would quickly understand everything there is to know about the physical world and just move on meta-physics. It would just be like an "orb".

Humans are already bored with the physical world and thus much prefer being in virtual spaces, look at everyone just starting at Instagram.

IMO This idea that it's going to eat all the atoms on Earth is part of the anthropomorphize of something mythological. Exactly how we imagine God as a dude with a grey beard.

Re: If AI scaling is to be shut down, let it be for a coherent reason

#470

Post by WorldPeas »

thomasahle wrote: post #237
WorldPeas wrote: post #28

Sure let's ban it, only to find out 6 months later that each of these companies simply obscured the development to try and get an edge. oh no! a 5 million(etc.) fine? too bad that's nowhere near how much profit their product will incur. Life goes on.

There's no way these companies could keep such development secret. Too many leakers in the inside.

Well yes, there would be leaks, but the fines resulting from them would not be appropriate for the violation
Post reply on HN