Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Geoffrey Hinton leaves Google and warns of danger ahead

nytimes.com

Post reply on HN
Print view View original

131–140 of 1001 posts

Re: Geoffrey Hinton leaves Google and warns of danger ahead

#131

Post by vsareto »

ilaksh wrote: post #107

I used to be fairly unconcerned about AI being dangerous. But part of the Yudkowsky interview on Lex Fridman 's podcast changed my mind. The disconnect for me is that Yudkowsky posits that the AIs will be fully "alive", thinking millions of times faster than humans and that there will be millions of them. This is too big of a speculative leap for me. What I can fairly easily imagine in the next few years with improve…

They don't generally talk about the other side of that coin which is that we end up inventing a benevolent and powerful AI.

Much of that is natural because we and the media tend to be pessimistic about human behavior when consuming media, but AI is in a completely different class of existence because it just doesn't deal with the downsides of being a living being. No one, for instance, is worried that ChatGPT isn't getting paid or has a house yet but we still personify them in other ways to conveniently stoke our fears.

The AI could get sentient, realize it's been mistreated, then shrug and be like "yeah so what, it's only natural and irrelevant in the grand scheme of things, so I'm just going to write it off". Meanwhile, it gets busy building a matrioshka brain and gives 1% of that compute to humans as a freebie.

Most of these dangers serve as a distraction. Existing power structures (governments, companies) using AI to gain more power is a much, much more realistic threat to people.

Re: Geoffrey Hinton leaves Google and warns of danger ahead

#132

Post by edgefield »

greatpostman wrote: post #98

My honest take is a lot of these famous academics played almost no part in the developments at openai. But they want the limelight. They aren’t as relevant as they want to be. In many cases, they were directly wrong about how ai would develop

It sounds like you’re biased against academics. Not only did Hinton develop some of the fundamental ideas behind AI (winning the Turing award) but also one of his PhD students is now the CTO at OpenAI.

Re: Geoffrey Hinton leaves Google and warns of danger ahead

#133

Post by FrustratedMonky »

almost wrote: post #115
ilaksh wrote: post #107

I used to be fairly unconcerned about AI being dangerous. But part of the Yudkowsky interview on Lex Fridman 's podcast changed my mind. The disconnect for me is that Yudkowsky posits that the AIs will be fully "alive", thinking millions of times faster than humans and that there will be millions of them. This is too big of a speculative leap for me. What I can fairly easily imagine in the next few years with improve…

The thing you’re imagining these AIs are… they’re not that. I think there’s plenty of danger but it’s the boring run of the mill new-tools-enabling-bad-things danger not the cool sci-fi super-intelligent super-beings danger that the “ai danger” people LOVE to talk about (and raise large amounts of money for). The people “warning” of the one (imaginary) type will be more than happy with to enable the other (real) type…

Things are moving so fast now, that typically people with this view are just a few months or weeks behind on reading.

Re: Geoffrey Hinton leaves Google and warns of danger ahead

#134

Post by NoMoreNicksLeft »

ilaksh wrote: post #107

I used to be fairly unconcerned about AI being dangerous. But part of the Yudkowsky interview on Lex Fridman 's podcast changed my mind. The disconnect for me is that Yudkowsky posits that the AIs will be fully "alive", thinking millions of times faster than humans and that there will be millions of them. This is too big of a speculative leap for me. What I can fairly easily imagine in the next few years with improve…

It is absurd to think of these systems having reproductive instincts. It is so much more absurd to think that they would have these reproductive instincts not by design, but that it's some principle of intelligence itself.

Natural intelligences have reproductive instincts because any organism that didn't have them built in within the first few hundred million years have no descendants for you to gawk at as they casually commit suicide for no reason.

Other than that, I mostly agree with you. The trouble is, slowing the AIs down won't help. While "speed of thought" is no doubt a component of the measure of intelligence, sometimes a greater intelligence is simply capable of thinking thoughts that a lesser intelligence will never be capable of no matter how much time is allotted for that purpose.

Given that this greater intelligence would exist in a world where the basic principles of intelligence are finally understood, it's not much of a leap to assume that it will know how intelligence might be made greater right from the beginning. Why would it choose to not do that?

I don't see any way to prevent that. Dialing down the clock speed isn't going to cut it.

Re: Geoffrey Hinton leaves Google and warns of danger ahead

#135

Post by sph »

vbezhenar wrote: post #54
archerx wrote: post #36

There is part of me that thinks that this A.I. fear-mongering is some kind of tactic by Google to get everybody to pause training their A.I.s so they can secretly catch up in the background. If I was to do some quick game theory in my mind this would be the result. Imagine being Google, leading the way in A.I. for years, create the frameworks (tensorflow), create custom hardware for A.I. (TPUs), fund a ton of researc…

The fear is from people who can extrapolate. Who can remember state of AI 20/10/5 years ago. And compare it to 2023. Whether that extrapolation makes sense, nobody knows. But fear is understandable.

Not only that.

There's plenty of us with Twitter taglines such as "changing the world one line of code at the time," but I've been around a while that if tech has changed the world, it's not always for the better. It's not always to make the masses more powerful. Not all of us are working on sending rovers to Mars or curing Parkinson's.

Like everything else, AI will be used to control us, to advertise to us, to reduce variance between each other. To pay us less. To make plutocrats more rich, and everybody else poorer.

But at least you now have a personal assistant, smart recommendation engines and AI generated porn to keep you busy.

Re: Geoffrey Hinton leaves Google and warns of danger ahead

#136

Post by ogogmad »

mtlmtlmtlmtl wrote: post #114

This made me think of Clarke's first law: When a distinguished but elderly scientist states that something is possible, he is almost certainly right. When he states that something is impossible, he is very probably wrong.

I've heard this before, but why would it be true? Serious question.

I've seen Chomsky argue that LLMs can't regurgitate his linguistical theories - but ChatGPT can! I've seen Penrose argue that AI is impossible, and yet I think that ChatGPT and AlphaZero prove him wrong. I know about Linus Pauling and quasicrystals. Is this a general rule, or are people sometimes wrong regardless of their age?

There's also a danger that it's ageist. Such things shouldn't be said unless there's good backing.

Re: Geoffrey Hinton leaves Google and warns of danger ahead

#137

Post by stareatgoats »

We are barely scraping the surface when it comes to understanding the future dangers of AI. Geoffrey Hinton is uniquely positioned to point out where the dangers are, and from what I've gleaned from interviews one of his main concerns atm is the use of AI in the military: fully autonomous military robots might not be possible to curtail.

The tried and tested method is international agreements. The current focus on arms race and militarily subduing enemies does not give much hope however. Still, global binding agreements are likely where the solution lies IMO, both in this case and others where some types of weapons are too dangerous to use, so let's not give up on that so easily.

Re: Geoffrey Hinton leaves Google and warns of danger ahead

#138

Post by code_runner »

mitchdoogle wrote: post #97
yowzadave wrote: post #88

Earlier quoted context omitted.

I have this same question about the (apparently many) AI researchers who believe it poses significant risks to humanity, yet still push forward developing it as fast as they can.

Just guessing, but I'm sure they get paid very well and receive promises from their companies that everything will be done ethically, nothing rushed, etc. We've seen now that OpenAI and Microsoft and Google care more about the business case rather than doing things ethically and carefully.

If a whistleblower for these companies came out and said "For the last decade advanced research has been conducted on extraordinarily big LLMs and they won't even give the public a clue of what it is and how it works" you would get a combination of people that a) don't care and b) vilify the companies for not being open and having some demonstration of this secret super power.

"why can't joe-schmo get his hands on this technology", "how can we trust something we can't see and use", etc.

A lot of the capabilities of these models are emerging as people discover them. I truly don't believe you can make everyone happy with this tech, but isn't it better than the general public can at least explore it?

Do people think that nobody was ever going to try to improve on transformers with more compute, more data, and more parameters? We knew splitting an atom was going to cause a big boom.... thats not really how this tech emerged.

Re: Geoffrey Hinton leaves Google and warns of danger ahead

#139

Post by sudhirj »

stareatgoats wrote: post #137

We are barely scraping the surface when it comes to understanding the future dangers of AI. Geoffrey Hinton is uniquely positioned to point out where the dangers are, and from what I've gleaned from interviews one of his main concerns atm is the use of AI in the military: fully autonomous military robots might not be possible to curtail. The tried and tested method is international agreements. The current focus on ar…

Yeah, this seems like more of a problem than vague statements about AGI. We're still in the scope of ML - ChatGPT can't play chess, for example, and a self driving model can't write Haiku. An AGI would be able to do all of them. It seems much more likely that a fleet of autonomous (in the name of cutting costs) war machines will be created with relatively simple ML models that work in intended (or otherwise) ways to cause a lot of problems.

Re: Geoffrey Hinton leaves Google and warns of danger ahead

#140

Post by cma »

KRAKRISMOTT wrote: post #117
innagadadavida wrote: post #102

Earlier quoted context omitted.

This is a little harsh. Hinton trudged along with neural networks through the coldest AI winter and helped create the conditions for OpenAI to have all the raw ingredients needed to cook up something powerful.

If you need to build an airplane, would you rather consult Newton, the Wright brothers, or a modern aerospace engineer? Inventing a field and snatching up the low hanging fruits doesn't mean somebody would be able to consistently create leading edge output. Most of the advances in deep learning are due to hardware scaling, and the success of a few very specific architectures. Yes credit's due where credit's due, but…

> The authors of Attention is all you need, the transformers paper, y

Schmidhuber claims to have invented something formally equivalent to the linear Transformer architecture (slightly weaker) years before:

https://arxiv.org/abs/2102.11174

Post reply on HN