Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

If AI scaling is to be shut down, let it be for a coherent reason

scottaaronson.blog

Post reply on HN
Print view View original

451–460 of 475 posts

Re: If AI scaling is to be shut down, let it be for a coherent reason

#451

Post by d00wgnir »

echelon wrote: post #332
PaulDavisThe1st wrote: post #327

Earlier quoted context omitted.

> Does it gain access to nukes? No, it becomes part of the decision-making process for deciding whether to launch, as well as part of the analysis system for sensor data about what is going out in the world. Just like social engineering is the best security hack, these new systems don't need to control existing systems, they just need to "control" the humans who do.

And is it there yet? Does ChatGPT have its finger on the trigger? I think everyone in the danger community is crying wolf before we've even left the house. That's just as dangerous. It's desensitizing everyone to the more plausible and immediate dangers. The response to "AI will turn the world to paperclips" is "LOL" The response to "AI could threaten jobs and may cause systems they're integrated into to behave unpre…

You only get one chance to align a created super intelligence before Pandora's box is opened. You can't put it back in the box. There may be no chance to learn from mistakes made. With a technology this powerful, it's never too early to research and prepare for the potential existential risk. You scoff at the "paperclips" meme, but it illustrates a legitimate issue.

Now, a reasonable counterargument might be that this risk justifies a limited amount of attention and concern, relative to other problems and risks we are facing. That said, the problem and risk are real, and there may be no takebacks. Preparing for tail risks are what humans are worst at. I submit that all caution is warranted, for both economic uncertainty and "paperclips"

Re: If AI scaling is to be shut down, let it be for a coherent reason

#452

Post by zirgs »

JohnFen wrote: post #410
zirgs wrote: post #404

Earlier quoted context omitted.

Usain Bolt can run faster than me. What's the point of exercising?

More like "what's the point of racing Usain Bolt?"

People are still playing chess despite the fact that a mobile phone can easily beat the world champion.

There are millions of people who are better programmers than me, but somehow I still have a job.

Re: If AI scaling is to be shut down, let it be for a coherent reason

#453

Post by nradov »

1attice wrote: post #432
nradov wrote: post #394

Earlier quoted context omitted.

The actual science never supported mask mandates. https://doi.org/10.1002/14651858.CD006207.pub6 When you don't know the proper course of action it's better to gather more data instead of taking swift but wrong actions.

The link I provided actually responds to the link you provided in response. Did you read it?

I read it but it was not a valid response.

Re: If AI scaling is to be shut down, let it be for a coherent reason

#454

Post by kypro »

carapace wrote: post #424
kypro wrote: post #420

Earlier quoted context omitted.

Comments like this make me confident that we're doomed. I mean no offence, but this is such an unthoughtful take that I don't even know where to start. > there is no cost to the GAI for allowing us to continue to exist. Of course there's a cost? Do you and the other 9 billion humans not consume resources? Why the hell would an AGI care about protecting all the worthless biological junk that clutters much of the Earth…

> I mean no offence (sic) None taken. > such an unthoughtful take No offense, but if you were 10 in 1999 when Iron Giant came out then I have been thinking about this for longer than you have been alive. > Of course there's a cost? Well, effectively zero as compared to the open-ended benefit of keeping pristine humanity around. Study ecology. There is no overlap between the ecological niches of non-biological sentien…

Well, I like this comment better, and for the record I have no problem with you being condescending, so please speak plainly as need be.

> Well, effectively zero as compared to the open-ended benefit of keeping pristine humanity around.

Could you expand? What is the benefit of keeping 9 billion resource hungry inferior life forms around and all of the biological matter needed to support them?

> Study ecology. There is no overlap between the ecological niches of non-biological sentience and humans. Ergo there is no reason for them to consume us. Our existence is orthogonal to theirs.

I understand what you're saying. It's not that there's an overlap in the ecological niches, it's that there is limited space and supporting billions of humans who want to drive cars, fly planes and live in cities requires significant space and resources.

To be clear though, I'm not arguing that they would have to destroy us. Humans didn't have to deforest the UK and wipe out the wolfs to survive. But given enough time their priorities will come first as did ours. Greenland will be replaced with solar farms and inconvenient human habits will be bulldozered.

> These things take up approximately zero space. The entire volume of biologically active matter is thinner relatively than the paint on a globe. In terms of the volume of just the solar system we are a "rounding error". There is plenty of room, plenty of energy, plenty of atoms.

Yet if you look at the Earth from space its landmass almost exclusively green from trees and plants, not grey from masses of server and solar farms. Depth isn't a very relevant metric here, it's about how the surface area is used.

Re: If AI scaling is to be shut down, let it be for a coherent reason

#456

Post by 93po »

the_af wrote: post #429
93po wrote: post #342

Earlier quoted context omitted.

This wouldn't be as effective as you think. A super intelligent AI can manipulate humans just fine/hold their family hostage/blackmail people.

I in no way want to seem like I endorse EY's brand of crackpotery, but do note he says in the interview a malicious AI would go undetected and would not try persuading humans to do anything to "get out of jail"; it would instead hack the system. EY seems to think a hostile AI would not contact human proxies in order to start its nefarious plans, because we'd be too slow for it, and also because alerting any of us wou…

For getting "out of jail", sure. But considering how we're already connecting it to the internet, and I doubt that is going away, it seems like a moot point. For other nefarious plans, I think it can contact human proxies without anyone thinking it was actually an AGI. Perfectly capable of sending emails and signing it as a John.

Re: If AI scaling is to be shut down, let it be for a coherent reason

#457

Post by osigurdson »

bombcar wrote: post #220
osigurdson wrote: post #180

Earlier quoted context omitted.

This may be in the $10^7 category now, but is there any reason to believe it will never be $10^3? Oddly the most pressing concern is “increased productivity”.

Unless it has something like the intentional self-latching of bitcoin mining, I do not see how it wouldn't rapidly drop in price. And if the models can be built once and then distributed, then it will certainly leak at some point, even if just intentionally by a hostile actor.

I can’t see how it could be conceived To work like bitcoin. The only reason that works is because a majority of humans agree to use the same code.

Re: If AI scaling is to be shut down, let it be for a coherent reason

#458

Post by riffraff »

slowmovintarget wrote: post #348
gspencley wrote: post #94

Earlier quoted context omitted.

You don't even have to look at other "fads" like 3D televisions and VR to be skeptical of the claim that recent advancements in ML* represent a major technological revolution that will change society like the Internet or the Printing Press did. Just look at ML's own history. A few years ago we had the "deepfake" scare. People were terrified that now videos are going to surface of politicians saying and doing things t…

This is not a fad. This is like the advent of writing. Plato opined that people would lose their facilities for memorization and the passing of oral histories would falter and die. He was correct about that. We don't have the same techniques and same facilities to memorize the Iliad verse for verse, epithet for epithet as the ancients did. He was incorrect about that being as ruinous as he imagined, but it was as dra…

> We don't have the same techniques and same facilities to memorize the Iliad verse for verse, epithet for epithet as the ancients did.

it's rhymes and metric, and I'm pretty sure in ancient time people got repetitions wrong too, which is why we have stories with variations everywhere in the world. What we miss is dedication, time and a facility with the language.

There's plenty of Italian people who have memorized Dante's Divine Comedy, for example.

Re: If AI scaling is to be shut down, let it be for a coherent reason

#459

Post by trogdor »

computerex wrote: post #405
joenot443 wrote: post #317

Earlier quoted context omitted.

There are PhDs who've spent their careers studying AI safety. It's a bit insulting and reductive to cast their work as "coming from science fiction", especially when it sounds like you haven't done much research on the topic.

There are PhD's who've spent their careers on string theory too with nothing to show for it. Powerful and bold claims require proportionally strong evidence. A lot of the FUD going around precludes that AGI means death. It's missing all logical steps and reasoning to establish this position. It's FUD at its core.

> A lot of the FUD going around precludes that AGI means death.

Just a friendly heads-up that “preclude” means “prevent,” or “make impossible.” I think you meant to say “assumes.”

Re: If AI scaling is to be shut down, let it be for a coherent reason

#460

Post by kilotaras »

rolisz wrote: post #445
jdiez17 wrote: post #334

Earlier quoted context omitted.

> My problem is with the first step: the existence of a super intelligent AI. Why are we sure it can exist? It's difficult to prove that something that has never been done before is possible, until it has been done. I personally don't see any fundamental limitations that would limit non-biological intelligence to human-level intelligence, for your preferred definition of intelligence. > And why we are we so sure GPT-…

I think that the moment you scale up the hardware for intelligence (from the 20cm brain) to a datacenter scale (to several hundred meters) you start getting into some other kinds of issues, such as it's hard to get a consistent view across the whole "brain". Synchronizing things will take longer, limited by the speed of light: suddenly things on the far ends can communicate with a latency with a lower bound of 100ns.…

> But we do have super human chess engines, but those are made in a completely different way from GPT.

Suppose GPT-X just dropped and it could both generate text AND play chess better than Magnus Carlssen.

How would that change you position on it being unable to pass the human-level intelligence barrier?

Post reply on HN