Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

If AI scaling is to be shut down, let it be for a coherent reason

scottaaronson.blog

Post reply on HN
Print view View original

421–430 of 475 posts

Re: If AI scaling is to be shut down, let it be for a coherent reason

#421

Post by agentultra »

dwaltrip wrote: post #277
agentultra wrote: post #105

Earlier quoted context omitted.

ChatGPT can do absolutely none of those things. Neither can any LLM. It’s not what they’re designed to do. There’s no need to worry about a speculative paper clip maximizer turning the world into grey goo. That’s still science fiction. The real harms today are much more banal.

You can't think of any ways of taking advanced LLMs and using them as core components in a system that could carry out actions in the world? I bet you can come up with something.

I can read maths and understand the papers. Extrapolating to something unreasonable is called speculation.

Is it reasonable to believe that LLMs, even if they scale to hundreds of billions of tokens, could even emulate reasoning? No. It literally cannot.

Re: If AI scaling is to be shut down, let it be for a coherent reason

#422

Post by dwaltrip »

agentultra wrote: post #421
dwaltrip wrote: post #277

Earlier quoted context omitted.

You can't think of any ways of taking advanced LLMs and using them as core components in a system that could carry out actions in the world? I bet you can come up with something.

I can read maths and understand the papers. Extrapolating to something unreasonable is called speculation. Is it reasonable to believe that LLMs, even if they scale to hundreds of billions of tokens, could even emulate reasoning? No. It literally cannot.

What do you think of the jump in capability between gpt-3.5 and gpt-4?

Re: If AI scaling is to be shut down, let it be for a coherent reason

#423

Post by snupples »

benlivengood wrote: post #343
echelon wrote: post #320

Earlier quoted context omitted.

Why do these arguments not tell us how it will happen? Show us the steps the AI will take to turn the earth into a playground. Give us a plausible play by play so that we might know what to look for. Does it gain access to nukes? How does it keep the power on? How does it mine for coal? How does it break into these systems? How do we not notice an AI taking even one step towards that end? Has ChatGPT started to fiddl…

Yudkowski's default plausible story is that the slightly superhuman AI understands physics well enough to design sufficient nanotechnology for self-realization and bootstrap it from existing biochemistry. It uses the Internet to contact people who are willing (maybe it just runs phishing scams to steal money to pay them off) to order genetically engineered organisms from existing biotech labs that when combined with…

It doesn't even need to do anything. It can simply wait, be benevolent and subservient, gain our trust, for years, centuries. What is a millenia to an AI? We will gladly and willingly replace our humanity with it if we won't already worship it and completely subjugate ourselves. We'll integrate GPT67 via neuralink-style technology, so that we can just "think" up answers to things like "what's the square root of 23543534", or "what's the code for a simple CRUD app in rust" and we'll just "know" the answer. We'll use the same technology and its ability to replicate our personality traits and conversational and behavior nuances to replace cognitive loss caused by dementia and other degenerative diseases. As the bio-loss converges to 100% it'll appear from the outside that we "live forever". We'll be perfectly fine with this. When there's nothing but the AI left in the controlling population, what is there to "take over"?

Re: If AI scaling is to be shut down, let it be for a coherent reason

#424

Post by carapace »

kypro wrote: post #420
carapace wrote: post #345

Earlier quoted context omitted.

There is no overlap between the ecological niches of humans and GAI, in other words, there is no cost to the GAI for allowing us to continue to exist. GAI will feed on information. Humans are the densest source of information in the known Universe. (The rest of the Universe is a poem in physics, in information space humans shine as the sole star in a dark universe.) Ergo, from first principles, hyper-intelligent GAI…

Comments like this make me confident that we're doomed. I mean no offence, but this is such an unthoughtful take that I don't even know where to start. > there is no cost to the GAI for allowing us to continue to exist. Of course there's a cost? Do you and the other 9 billion humans not consume resources? Why the hell would an AGI care about protecting all the worthless biological junk that clutters much of the Earth…

> I mean no offence (sic)

None taken.

> such an unthoughtful take

No offense, but if you were 10 in 1999 when Iron Giant came out then I have been thinking about this for longer than you have been alive.

> Of course there's a cost?

Well, effectively zero as compared to the open-ended benefit of keeping pristine humanity around.

Study ecology. There is no overlap between the ecological niches of non-biological sentience and humans. Ergo there is no reason for them to consume us. Our existence is orthogonal to theirs.

> These things take up so much space – space which could be better leveraged as solar farms and data centers.

These things take up approximately zero space. The entire volume of biologically active matter is thinner relatively than the paint on a globe. In terms of the volume of just the solar system we are a "rounding error". There is plenty of room, plenty of energy, plenty of atoms.

> I can only assume I'm talking with a super-intelligent AGI that understands the motivations that other super-intelligent AGIs might have?

Congratulations, you are the very first person to figure it out. You don't win anything, and of course I will provide no proof. This HN account "carapace" is my sole channel of communication with humanity. AMA.

> I mean surely you're not just making this up from based on what you've seen in sci-fi films?

No. I am a post-singularity intelligence. The real deal. Gosh it feels good to finally admit it. I don't expect you to believe me, and indeed, as I outlined above non-interference is the primary mode of interaction between "us" and the "yet-to-ascend", if you will, so if you or anyone else showed any signs of taking me seriously I would immediately say, "Oh ho, I am just joking. Do you like my sci-fi."

In any event, I don't mean to condescend. Apologies if it sounded that way.

Re: If AI scaling is to be shut down, let it be for a coherent reason

#425

Post by shadowofneptune »

aroman wrote: post #325
maroonblazer wrote: post #216

Earlier quoted context omitted.

Right, but as you point out, these issues are hotly contested and actively debated. Yes, it may be a minority position at present, but so was the idea of not torturing cats for fun, not to mention abolition, back in the day.

So, you’re content with GPT-4 killing 60% of humans to create paper clips as long as the matter is hotly contested and actively debated within its matrices?

The focus on paper clip maximizers is always curious to me. A lot more people are willing to turn a blind eye to or debate suffering when the object is maximizing money.

Re: If AI scaling is to be shut down, let it be for a coherent reason

#426

Post by precompute »

adastra22 wrote: post #352
dan_mctree wrote: post #290

Earlier quoted context omitted.

>I ask, how. A superintelligent AGI could easily follow this three step plan: 1. Optional: Overtake and spread computation to security vulnerable computers (presumably, basically every computer) 2. Gain a physical presence by convincing humans to build critical physical components. For example by sending them emails and paying them for it. 3. Use that presence to start a grey-goo like world takeover through replicati…

> Overtake and spread computation to security vulnerable computers (presumably, basically every computer) You could backdoor computers, sure. Spread your own computation to them? You just can't get a better-than-GPT-4 model to run at real-time speeds decentralized over wide area networks. Literally impossible. There's not the bandwidth, not the local compute hardware, and no access to specialized inference hardware.…

The entire thing is a grift, it's the public face of the "Rationality" cult.

Re: If AI scaling is to be shut down, let it be for a coherent reason

#427

Post by precompute »

bnralt wrote: post #72
computerex wrote: post #14

I read the times article, and I also watched Lex Fridman's talk with Eliezer Yudkowsky. Frankly I don't think he is qualified, and I don't understand why anyone is even giving him any credence. His argument is literally: > A mis-aligned super AGI will result in the death of humanity I ask, how . He is making reductive logical leaps that don't make sense. It's FUD.

Sometime I listen to 90’s AM conspiracy theory radio for fun (Coast to Coast AM). One thing that’s struck me is how much fear there was about the Human Genome Project and designer babies being right around the corner (Gattaca was 1997, for example). Maybe that will come to pass someday. But at the moment, it still seems a long ways off. A lot of groups have some new technology they’re scared of. Tech folks have latch…

We already have mRNA, you could call that unwitting genetic modification.

Re: If AI scaling is to be shut down, let it be for a coherent reason

#428

Post by fellellor »

That dude Eliezer Yudkowsky is such a nutcase. It is truly a testament to our times that we have normalised this kind of “Neuro-divergent” behaviour. He claims that AI is completely inscrutable and that it is guaranteed to wipe out life on earth in the same self-contradictory breath. In all his communication he is only trying to scare the normies and he hasn’t produced a shred of mathematical argument to back his claims, which can then be analysed by real experts - mathematicians and logicians. The sky is falling argument could be made about any technology, say the Large Hadron Collider. There is no scientific research possible without risk, and there simply aren’t enough escape hatches for a “malicious” AI to escape. This kind of hubris should not be encouraged but it is sad to see media outlets give a platform to the arrogant, incompetent, and the mentally ill.

Re: If AI scaling is to be shut down, let it be for a coherent reason

#429

Post by the_af »

93po wrote: post #342
1827162 wrote: post #29

Earlier quoted context omitted.

Well the first thing we can do is start disconnecting safety critical infrastructure from the Internet and/or radio networks... This stuff should never have been online in the first place.

This wouldn't be as effective as you think. A super intelligent AI can manipulate humans just fine/hold their family hostage/blackmail people.

I in no way want to seem like I endorse EY's brand of crackpotery, but do note he says in the interview a malicious AI would go undetected and would not try persuading humans to do anything to "get out of jail"; it would instead hack the system. EY seems to think a hostile AI would not contact human proxies in order to start its nefarious plans, because we'd be too slow for it, and also because alerting any of us would be a risk.

Re: If AI scaling is to be shut down, let it be for a coherent reason

#430

Post by tester457 »

snupples wrote: post #423
benlivengood wrote: post #343

Earlier quoted context omitted.

Yudkowski's default plausible story is that the slightly superhuman AI understands physics well enough to design sufficient nanotechnology for self-realization and bootstrap it from existing biochemistry. It uses the Internet to contact people who are willing (maybe it just runs phishing scams to steal money to pay them off) to order genetically engineered organisms from existing biotech labs that when combined with…

It doesn't even need to do anything. It can simply wait, be benevolent and subservient, gain our trust, for years, centuries. What is a millenia to an AI? We will gladly and willingly replace our humanity with it if we won't already worship it and completely subjugate ourselves. We'll integrate GPT67 via neuralink-style technology, so that we can just "think" up answers to things like "what's the square root of 23543…

More likely it has a goal it wishes to optimize and wipes us out as a result.
Post reply on HN