Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

If AI scaling is to be shut down, let it be for a coherent reason

scottaaronson.blog

Post reply on HN
Print view View original

111–120 of 475 posts

Re: If AI scaling is to be shut down, let it be for a coherent reason

#111

Post by [deleted] »

nl wrote: post #65
nradov wrote: post #56

Earlier quoted context omitted.

There are actually no relevant experts in the field of artificial general intelligence, or the safety thereof. No one has defined a clear path to build such a thing. Claiming to be an expert in this field is like claiming to be an expert in warp drives or time machines. Those calling for a halt in research are merely ignorant, or attention seeking grifters. Their statements can be dismissed out of hand regardless of…

> Current LLMs are merely sophisticated statistical tools. This is wrong. They are have the capability for in-context learning, which doesn't match most definitions of "statistical tools"

[deleted]

Re: If AI scaling is to be shut down, let it be for a coherent reason

#112

Post by nradov »

tibbon wrote: post #27
computerex wrote: post #14

I read the times article, and I also watched Lex Fridman's talk with Eliezer Yudkowsky. Frankly I don't think he is qualified, and I don't understand why anyone is even giving him any credence. His argument is literally: > A mis-aligned super AGI will result in the death of humanity I ask, how . He is making reductive logical leaps that don't make sense. It's FUD.

I don’t think it even requires that AI to be sentient or malicious. The humans already are. Given a tool for carnage and hatred, people will use it. How long did it take from us getting the atomic bomb working to use in production? Less than three months. Will Putin or terrorists hold back from using it in terrible ways if they have it available to them?

The printing press has been used to incite carnage and hatred for centuries. Should we have restricted printing presses until we figured out how to prevent them from being used for genocidal propaganda?

Re: If AI scaling is to be shut down, let it be for a coherent reason

#113

Post by judge2020 »

misssocrates wrote: post #100
nl wrote: post #11

Why are we even bothering talking about this? It's Gary Marcus "neural networks don't really work" suddenly discovering they do, and literally trying to shut down research in that area while keeping his prefered research areas funded. We know a bunch of the people whose names are on the letter didn't sign it (eg Yann Lecunn who said he disagreed with the premise and didn't sign it). I'm so offended by the idea of thi…

What if governments start regulating and locking down advanced computing in the same way they locked down medicine and advanced weapontry?

So far it seems letting private industry iterate on LLMs doesn't directly pose to a risk of ending lives like human trials and nuclear weaponry development do.

Re: If AI scaling is to be shut down, let it be for a coherent reason

#114

Post by silveroriole »

skocznymroczny wrote: post #54

Am I the only one who's not very concerned about ChatGPT and "AI" in general? I am young, but still lived through several hype phases. I remember when 3D TVs were going to be mainstream and 2D was considered legacy. I remember when PC was to die soon and to be replaced by smartphones. I remember when VR was to become as common accessory as a game controller. It's 2023, and I still don't have automated self driving ca…

> “remember that? yeah it was 5 years already”

I get the impression many HN commenters haven’t even been adults for 5 years so no, they really don’t remember it :) for example articles get posted here and upvoted with the author boasting about their insights from being in the software industry for 3 years!

Re: If AI scaling is to be shut down, let it be for a coherent reason

#115

Post by adamsmith143 »

nl wrote: post #75
NumberWangMan wrote: post #58

Earlier quoted context omitted.

I don't think Eliezer Yudkowsky is very good at bridging the gap with other people in conversations, because most people haven't thought about this as much. However, while it's terrifying and I hate it, and I keep trying to convince myself that he's wrong, I believe him. The first super-intelligent AI will be an alien kind of intelligence to us. It will not have any of the built-in physical and emotional responses we…

> An AI trained to end cancer might just figure out a plan to kill everyone with cancer. An AI trained to reduce the number of people with cancer without killing them might decide to take over the world and forcibly stop people from reproducing, so that eventually all the humans die and there is no cancer -- technically it didn't kill anyone! I don't understand this and other paperclip maximizer type arguments. If a…

Because if you don't find out a way for it to hold human values extremely well then an easy solution to "Cure All Cancer" is to "Kill all Humans", no Humans no Cancer. Without an explicit understanding that this is not an actually acceptable outcome for humans an AI will happily execute it. THAT is the fundamental problem, how do you get human values into these systems.

Re: If AI scaling is to be shut down, let it be for a coherent reason

#116

Post by lkbm »

dandellion wrote: post #92
lkbm wrote: post #37

Earlier quoted context omitted.

Academics, businessmen, and other non-governmental entities are usually better off advocating for the government to enact regulations than engaging in paramilitary action on their own. Do you think Extinction Rebellion is doing more to fight climate change than the people working with governments to enact good climate policy? Do you think PETA is helping animal well-fare more than more than the various organizations…

I don't think it's so strange to expect the reaction to be proportionate to the threat they see? An open letter seems several orders of magnitude shorter to anything that would be effective if it really is a threat to the species. I think of examples of ways people react when there are very high stakes: resigning in protest, going on a hunger strike, demonstrations/raids, setting themselves on fire, etc. But there's…

I agree there's a lot more than "sign an open letter" they could be doing. I'm mostly objecting to the "they need to engage in terrorism or they're not serious" assertion.

As for resigning in protest, my understanding is that Anthropic was founded by people who quit OpenAI saying it was acting recklessly. That seems like the best skin-in-the-game signal. I find that much more compelling than I would the Unabomber route.

People like Musk and Woz should probably be pouring money into safety research and lobbying, but I don't think a hunger strike from anyone would make a difference, and resigning only makes sense if you work for OpenAI, Google, or a handful of other places where most of these people presumably don't work.

What should I, an employee of a company not developing AI be doing? The only reasonable actions I can see are 1. work on AI safety research, 2. donate money to AI safety research/advocacy, and 3. sign and open letter.

(I did none of these, to be fair, and am even giving a monthly fee to OpenAI to use ChatGPT with GPT-4), but my partner, an AI researcher who is seriously concerned, tried to sign the letter pit was ratelimited at the time], and is considering working in AI safety post-graduation. If she weren't making a PhD-student's salary, she might be donating money as well, though it's not super-clear where to direct that money.)

Re: If AI scaling is to be shut down, let it be for a coherent reason

#117

Post by nl »

misssocrates wrote: post #100
nl wrote: post #11

Why are we even bothering talking about this? It's Gary Marcus "neural networks don't really work" suddenly discovering they do, and literally trying to shut down research in that area while keeping his prefered research areas funded. We know a bunch of the people whose names are on the letter didn't sign it (eg Yann Lecunn who said he disagreed with the premise and didn't sign it). I'm so offended by the idea of thi…

What if governments start regulating and locking down advanced computing in the same way they locked down medicine and advanced weapontry?

Well they do of course. There are export restrictions on supercomputers now, including many NVIDIA GPUs.

I contend that doesn't matter.

There is sufficient compute available now at consumer levels to make it too late to stop training LLMs.

If cloud A100s became unavailable tomorrow it'd be awkward, but there is enough progress being made on training on lower RAM cards to show it is possible.

Re: If AI scaling is to be shut down, let it be for a coherent reason

#118

Post by adamsmith143 »

spaceman_2020 wrote: post #25

A little off-topic, but does anyone wonder what kind of AI tech the CIA or DARPA or the NSA might have? Or what they might be building? It's a little scary that a small private company could create an AI that has dominated headlines for over a month now. Surely, the folks at NSA, who have more data than anyone else, have taken notice and are working on something of their own. Am I being paranoid about a potential too…

Its pretty simple, there aren't thousands of missing GPUs that would be required for an agency to be training massive models on and there aren't hundreds of the best PhD students disappearing while working somewhere in the DC Metro Area. They don't seem to have the hardware and they certainly don't seem to have the brainpower either.

Re: If AI scaling is to be shut down, let it be for a coherent reason

#119

Post by nl »

lkbm wrote: post #95
nl wrote: post #50

Earlier quoted context omitted.

I don't particularly care about crowds on Twitter or HN. Musk has lots of money but can't stop me spending mine. Marcus said: > We must demand transparency, and if we don’t get it, we must contemplate shutting these projects down. https://garymarcus.substack.com/p/the-sparks-of-agi-or-the-e... (While at the same time still saying "it's nothing to do with AGI")

Sure, sure, Marcus says one thing and has one set of motives. Elon Musk says another thing and has another set of motives. But if you want to dismiss this by questioning the motives of the signers, you've got a few thousand other people whose motives you have to identify and dismiss. It would be much more effective, and meaningful, to challenge their arguments.

I think the Scott Aaroson link posted did a pretty good job of that.

I don't think their arguments deserve anything more than that.

Re: If AI scaling is to be shut down, let it be for a coherent reason

#120

Post by mrob »

adamsmith143 wrote: post #115
nl wrote: post #75

Earlier quoted context omitted.

> An AI trained to end cancer might just figure out a plan to kill everyone with cancer. An AI trained to reduce the number of people with cancer without killing them might decide to take over the world and forcibly stop people from reproducing, so that eventually all the humans die and there is no cancer -- technically it didn't kill anyone! I don't understand this and other paperclip maximizer type arguments. If a…

Because if you don't find out a way for it to hold human values extremely well then an easy solution to "Cure All Cancer" is to "Kill all Humans", no Humans no Cancer. Without an explicit understanding that this is not an actually acceptable outcome for humans an AI will happily execute it. THAT is the fundamental problem, how do you get human values into these systems.

Even "kill all humans" is difficult to define. Is a human dead if you flash-freeze them in liquid helium? It would certainly make it easier to cut out the cancer. And nobody said anything about defrosting them later. And even seemingly healthy humans contain cancerous cells. There's no guarantee their immune system will get all of them.
Post reply on HN