Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

If AI scaling is to be shut down, let it be for a coherent reason

scottaaronson.blog

Post reply on HN
Print view View original

141–150 of 475 posts

Re: If AI scaling is to be shut down, let it be for a coherent reason

#141

Post by mrob »

dandellion wrote: post #92
lkbm wrote: post #37

Earlier quoted context omitted.

Academics, businessmen, and other non-governmental entities are usually better off advocating for the government to enact regulations than engaging in paramilitary action on their own. Do you think Extinction Rebellion is doing more to fight climate change than the people working with governments to enact good climate policy? Do you think PETA is helping animal well-fare more than more than the various organizations…

I don't think it's so strange to expect the reaction to be proportionate to the threat they see? An open letter seems several orders of magnitude shorter to anything that would be effective if it really is a threat to the species. I think of examples of ways people react when there are very high stakes: resigning in protest, going on a hunger strike, demonstrations/raids, setting themselves on fire, etc. But there's…

If a terrorist bombs one data center, security increases at all the other data centers. Bombing all data centers (and chip fabs, so they can't be rebuilt) simultaneously requires state-level resources.

Re: If AI scaling is to be shut down, let it be for a coherent reason

#142

Post by kajaktum »

agentultra wrote: post #57

We're not even talking about AGI or an entity that is "smart" somehow. It's a bloody algorithm. The danger has been the people who use it and find ways to abuse other people with it. This whole idea that we're going to suddenly have HAL 9000 or Terminators running around is mob hype mentality. The responsible thing to do here is to educate. AGI isn't likely to destroy humanity. Humanity is already doing a good job of…

The danger is socioeconomic. AI will displace many, many jobs (it already does). Many people will claim that we will simply create different, new jobs. However, we have to think about what kind of job ja now viable for the average person? Typewriter used to be a decent job. Now, kids is probably expected to know how to do it. Can we keep up? Folks at HN seems to overestimate what the general population is capable of.

Re: If AI scaling is to be shut down, let it be for a coherent reason

#143

Post by circlefavshape »

adamsmith143 wrote: post #115
nl wrote: post #75

Earlier quoted context omitted.

> An AI trained to end cancer might just figure out a plan to kill everyone with cancer. An AI trained to reduce the number of people with cancer without killing them might decide to take over the world and forcibly stop people from reproducing, so that eventually all the humans die and there is no cancer -- technically it didn't kill anyone! I don't understand this and other paperclip maximizer type arguments. If a…

Because if you don't find out a way for it to hold human values extremely well then an easy solution to "Cure All Cancer" is to "Kill all Humans", no Humans no Cancer. Without an explicit understanding that this is not an actually acceptable outcome for humans an AI will happily execute it. THAT is the fundamental problem, how do you get human values into these systems.

Other animals get cancer too.

Re: If AI scaling is to be shut down, let it be for a coherent reason

#144

Post by [deleted] »

r3trohack3r wrote: post #136
nl wrote: post #11

Why are we even bothering talking about this? It's Gary Marcus "neural networks don't really work" suddenly discovering they do, and literally trying to shut down research in that area while keeping his prefered research areas funded. We know a bunch of the people whose names are on the letter didn't sign it (eg Yann Lecunn who said he disagreed with the premise and didn't sign it). I'm so offended by the idea of thi…

This is my view too. The perceived value of LLMs relative to everything that came before is staggering. I wouldn’t expect any regulation or laws to slow it down at this point. You might get the U.S. to crack down on it and stop innovation, GPL or proprietary license holders might win a lawsuit, etc. But I suspect the net effect would be to push research and innovation out of the jurisdiction that made that decision a…

[deleted]

Re: If AI scaling is to be shut down, let it be for a coherent reason

#145

Post by natdempk »

cableshaft wrote: post #128
lukev wrote: post #36

Earlier quoted context omitted.

The reasoning pattern of Yud and his ilk is fundamentally theological and eschatological in nature. There isn't any actual understanding of the technology involved. It is fundamentally an ontological argument. Because they can imagine a god-like super intelligent AI, it must be possible. And they're associating that with LLMs because that's the most powerful AI currently available, not based on any actual capabilitie…

I suspect that he realized that there's money to be made being the professional luddite of the hot thing du jour that talk shows and schools can give him money for (as so many other people have done for other things), and he's taking this simple claim of 'AGI will murder us all' to get on as many talk shows as possible and make as much money off of it. I doubt he's speaking in public out of any concern for humanity's…

FWIW he’s been making these claims well before the current AI hype cycle hit.

Re: If AI scaling is to be shut down, let it be for a coherent reason

#146

Post by fnimick »

dangond wrote: post #132
maroonblazer wrote: post #66

Earlier quoted context omitted.

>I don’t think it even requires that AI to be sentient or malicious. The humans already are. We are and we aren't. I was struck by this line in the OP: >AI is manifestly different from any other technology humans have ever created, because it could become to us as we are to orangutans; As far as I can tell, we humans treat orangutans quite kindly. I.e., on the whole, we don't go around killing them indiscriminately o…

The whole point is that no one knows how to engineer these principles into a model, and no one has a good plan for doing so either.

Not to mention that having "principles" is going to handicap you in a competitive environment where not being on top means you might as well be last.

Re: If AI scaling is to be shut down, let it be for a coherent reason

#147

Post by dandellion »

lkbm wrote: post #116
dandellion wrote: post #92

Earlier quoted context omitted.

I don't think it's so strange to expect the reaction to be proportionate to the threat they see? An open letter seems several orders of magnitude shorter to anything that would be effective if it really is a threat to the species. I think of examples of ways people react when there are very high stakes: resigning in protest, going on a hunger strike, demonstrations/raids, setting themselves on fire, etc. But there's…

I agree there's a lot more than "sign an open letter" they could be doing. I'm mostly objecting to the "they need to engage in terrorism or they're not serious" assertion. As for resigning in protest, my understanding is that Anthropic was founded by people who quit OpenAI saying it was acting recklessly. That seems like the best skin-in-the-game signal. I find that much more compelling than I would the Unabomber rou…

Yes, engaging in terrorism would be too much for most signers in the list, but the point is more that there is a wide gap between what they're saying and what they're doing. You make another good point that at the very least, they should be putting their money where they mouth is.

Anthropic seem to be competing against OpenAI? And getting funds from Google? So they would probably benefit economically from delaying development, since they are currently behind. Personally I think it's more important to look at what people are doing, than just listening to what they say, as there is a strong tendency to posturing.

Re: If AI scaling is to be shut down, let it be for a coherent reason

#148

Post by pkoird »

mark_l_watson wrote: post #135

For me, this is complex. My first impression is that many of the signers work on older methods than deep learning and LLMs. Sour grapes. Of course, real AGI has its dangers, but as Andrew Ng has said, worrying about AGI taking over the world is like worrying about overcrowding of Mars colonies. Both tech fields are far in the future. The kicker for me though is: we live in an adversarial world, so does it make sense…

Far in the future? Just 6 months ago, people believed that ChatGPT like model would take 10-15 years more. I believe that Andrew himself doesn't really understand how LLMs work. In particular, what is about the increase in their parameters that induces emergence and what exactly is the nature of such Emergence. So yeah, AGI might be far into the future but it might just be tomorrow as well.

Re: If AI scaling is to be shut down, let it be for a coherent reason

#149

Post by timcobb »

quonn wrote: post #62
1827162 wrote: post #26

Earlier quoted context omitted.

Assuming the training is based on public data, then making it illegal is like a "thought crime" where the state is trying to prohibit carrying out certain computations. Well if I believe the government has no right to control my thoughts, then I believe it has no right to control whatever computation I do on my private computer as well. Down with these ** control freaks, so much of everything nowadays prioritizes saf…

As a hypothetical, if some computations are extremely dangerous for the general public, it should be possible to ban them.

Extremely dangerous computations

Re: If AI scaling is to be shut down, let it be for a coherent reason

#150

Post by deepsquirrelnet »

nl wrote: post #11

Why are we even bothering talking about this? It's Gary Marcus "neural networks don't really work" suddenly discovering they do, and literally trying to shut down research in that area while keeping his prefered research areas funded. We know a bunch of the people whose names are on the letter didn't sign it (eg Yann Lecunn who said he disagreed with the premise and didn't sign it). I'm so offended by the idea of thi…

> I'm so offended by the idea of this that I'll personally fund $10k of training runs myself in a jurisdiction where it isn't banned if this ever became law in the US.

And this is exactly the risk. These things will just go other places while the US trips over its salty billionaires trying to create political red tape to allow their own businesses to catch up.

Surely Elon has no reservations about putting premature self driving cars on the road. Or are we just going to pretend that the last decade of his business failures have nothing to do with this hypocrisy? At least GPT hasn’t killed anyone yet.

An open letter full of forged signatures and conflicts of interest aren’t convincing enough for the repercussions of the government stepping in. More than anything, this just reeks of playing on peoples fear of change.

New things are scary, but in my opinion, this kind of drastic measure starts with broad interest panels that first evaluate and detail the risks. Ooo scary doesn’t cut it.

Post reply on HN