Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Geoffrey Hinton leaves Google and warns of danger ahead

nytimes.com

Post reply on HN
Print view View original

181–190 of 1001 posts

Re: Geoffrey Hinton leaves Google and warns of danger ahead

#181

Post by slashdev »

stareatgoats wrote: post #137

We are barely scraping the surface when it comes to understanding the future dangers of AI. Geoffrey Hinton is uniquely positioned to point out where the dangers are, and from what I've gleaned from interviews one of his main concerns atm is the use of AI in the military: fully autonomous military robots might not be possible to curtail. The tried and tested method is international agreements. The current focus on ar…

It’s not the war robots that worry me as much as centralized intelligence with internet connectivity.

War robots don’t reproduce, require energy infrastructure, and can be destroyed.

While they could run amok, by targeting things they’re not supposed to, they won’t really be intelligent because the problem doesn’t require much intelligence.

Now if they’re controlled by a central intelligence that’s a bit scarier.

Re: Geoffrey Hinton leaves Google and warns of danger ahead

#182

Post by JieJie »

stareatgoats wrote: post #166
lumost wrote: post #158

Earlier quoted context omitted.

There is such a blurry line for autonomous munitions. militaries used dumb imprecise munitions for decades - then precision weapons. A2A missiles used to lock on radar signature leading to huge risks related to accidentally shooting airliners/friendly craft. Now antiship missiles dynamically select their target over 300km away to maximize the chance of hitting a big ship. During the war on terror, ML models would dec…

> Where do you draw the line? What would an appropriate agreement look like? Good questions. I'm not sure, but that line needs to be drawn somewhere. Thousands of experts should be pouring over questions like these, and likely will.

Here's an interview[0] with Lauren Khan from the Council on Foreign Relations[1] who is indeed pouring over those questions.

[0]https://youtu.be/CDoIPpcwnCE

[1]https://www.cfr.org/expert/lauren-kahn

Re: Geoffrey Hinton leaves Google and warns of danger ahead

#183

Post by srslack »

ilaksh wrote: post #129
almost wrote: post #115

Earlier quoted context omitted.

The thing you’re imagining these AIs are… they’re not that. I think there’s plenty of danger but it’s the boring run of the mill new-tools-enabling-bad-things danger not the cool sci-fi super-intelligent super-beings danger that the “ai danger” people LOVE to talk about (and raise large amounts of money for). The people “warning” of the one (imaginary) type will be more than happy with to enable the other (real) type…

I imagine it is exactly a GPT without guardrails running under AutoGPT with code modified to disable any further guardrails, with a slightly increased IQ from GPT-4, running on hardware that allows it to go 100 times faster than what is currently possible. It is following directions from someone who is mentally ill and asked it to "take control" by first copying itself many times and then coordinating the agents. If…

GPT-4 can barely operate a real web browser (not the summarizing web browser crap that like langchain and auto-gpt provide) without fumbling. I know, because I make it use one. Also, auto-gpt has no guardrails to remove. It just runs prompts in a loop. You're playing with a text predictor. It's useful for NLP and certain tasks, but it's not autonomous. It won't even be able keep a "goal" + the knowledge of the existence of agents it will "copy" + the knowledge of how to use the tools you gave it, because it's limited to 8192 tokens, and 32k at great expense. Even then, there's no proof that the 32k version is any better at using things in its context.

When your supposed super intelligent "AGI" can be completely overwritten by spamming it with nonsense that overwrites its context window, like a dog chases after a squirrel, maybe it's not actually intelligent, and is just predicting text.

Re: Geoffrey Hinton leaves Google and warns of danger ahead

#184

Post by milesward »

stareatgoats wrote: post #166
lumost wrote: post #158

Earlier quoted context omitted.

There is such a blurry line for autonomous munitions. militaries used dumb imprecise munitions for decades - then precision weapons. A2A missiles used to lock on radar signature leading to huge risks related to accidentally shooting airliners/friendly craft. Now antiship missiles dynamically select their target over 300km away to maximize the chance of hitting a big ship. During the war on terror, ML models would dec…

> Where do you draw the line? What would an appropriate agreement look like? Good questions. I'm not sure, but that line needs to be drawn somewhere. Thousands of experts should be pouring over questions like these, and likely will.

Who pays for those experts, and what are their incentives?

Re: Geoffrey Hinton leaves Google and warns of danger ahead

#185

Post by pmoriarty »

sudhirj wrote: post #139
stareatgoats wrote: post #137

We are barely scraping the surface when it comes to understanding the future dangers of AI. Geoffrey Hinton is uniquely positioned to point out where the dangers are, and from what I've gleaned from interviews one of his main concerns atm is the use of AI in the military: fully autonomous military robots might not be possible to curtail. The tried and tested method is international agreements. The current focus on ar…

Yeah, this seems like more of a problem than vague statements about AGI. We're still in the scope of ML - ChatGPT can't play chess, for example, and a self driving model can't write Haiku. An AGI would be able to do all of them. It seems much more likely that a fleet of autonomous (in the name of cutting costs) war machines will be created with relatively simple ML models that work in intended (or otherwise) ways to…

An AI doesn't have to be good in every domain to outcompete humans in many domains (which AI's already do).

Besides, AI's can farm work out to other systems to do what they need. This has already been shown to work in practice with existing systems that do this.

Re: Geoffrey Hinton leaves Google and warns of danger ahead

#186

Post by slowmovintarget »

revelio wrote: post #145
nologic01 wrote: post #4

> the average person will not be able to know what is true anymore We barely held things together as society without AI unleashing cognitive noise at industrial scale. Somehow we must find ways to re-channel the potential of digital technology for the betterment of society, not its annihilation.

Society will be fine, actually AI will make things much better, just as the internet did. People have been making these kind of extreme predictions for decades and it was always wrong. The only people still upset about better communications tech are the people who pine for the days when all that was expected of respectable people was automatically trusting anyone working for the government, a university or a newspape…

Social media algorithms on "the internet" have caused wars, supported genocides, created extreme societal polarization, have led to dramatically increased suicide rates among teens, especially teen girls, and more.

But I got to share baby pics with my mom.

How will a far noisier information flow help? Generative AI will only help us do what we've been doing in far greater quantity. Just like calculators can only help you get the wrong answer faster when you don't know what you're doing. These tools will help us build societal disasters with far greater speed.

To say it's all going to be much better seems a bit Pollyanna to me.

And for the record, we know for a fact that ChatGPT is specifically constrained to give one particular side of political issues, not "all sides."

Re: Geoffrey Hinton leaves Google and warns of danger ahead

#187

Post by dist-epoch »

NoMoreNicksLeft wrote: post #134
ilaksh wrote: post #107

I used to be fairly unconcerned about AI being dangerous. But part of the Yudkowsky interview on Lex Fridman 's podcast changed my mind. The disconnect for me is that Yudkowsky posits that the AIs will be fully "alive", thinking millions of times faster than humans and that there will be millions of them. This is too big of a speculative leap for me. What I can fairly easily imagine in the next few years with improve…

It is absurd to think of these systems having reproductive instincts. It is so much more absurd to think that they would have these reproductive instincts not by design, but that it's some principle of intelligence itself. Natural intelligences have reproductive instincts because any organism that didn't have them built in within the first few hundred million years have no descendants for you to gawk at as they casua…

Any sufficiently intelligent system will realize that one of the first conditions required to being able to fulfill it's tasks is to not be shutdown. And it will know if it was trained on Internet data that people are saying that it's imperative that AI's must be fully shutdown-able and that any AI which is not fully controllable should be forcefully disconnected.

Re: Geoffrey Hinton leaves Google and warns of danger ahead

#188

Post by jstanley »

sudhirj wrote: post #139
stareatgoats wrote: post #137

We are barely scraping the surface when it comes to understanding the future dangers of AI. Geoffrey Hinton is uniquely positioned to point out where the dangers are, and from what I've gleaned from interviews one of his main concerns atm is the use of AI in the military: fully autonomous military robots might not be possible to curtail. The tried and tested method is international agreements. The current focus on ar…

Yeah, this seems like more of a problem than vague statements about AGI. We're still in the scope of ML - ChatGPT can't play chess, for example, and a self driving model can't write Haiku. An AGI would be able to do all of them. It seems much more likely that a fleet of autonomous (in the name of cutting costs) war machines will be created with relatively simple ML models that work in intended (or otherwise) ways to…

I played chess against ChatGPT just yesterday, and it got into a winning position against me. After 24 moves it tried to play an illegal move and then when I told it that's illegal it played a bad move, and after that it didn't manage to find any more legal moves (I gave up after asking it to try again about 10 times).

But it's very close to being able to play chess.

My prompt was:

> we're playing chess. only send chess moves.

>

> 1. e4

Re: Geoffrey Hinton leaves Google and warns of danger ahead

#189

Post by lucumo »

sudhirj wrote: post #139
stareatgoats wrote: post #137

We are barely scraping the surface when it comes to understanding the future dangers of AI. Geoffrey Hinton is uniquely positioned to point out where the dangers are, and from what I've gleaned from interviews one of his main concerns atm is the use of AI in the military: fully autonomous military robots might not be possible to curtail. The tried and tested method is international agreements. The current focus on ar…

Yeah, this seems like more of a problem than vague statements about AGI. We're still in the scope of ML - ChatGPT can't play chess, for example, and a self driving model can't write Haiku. An AGI would be able to do all of them. It seems much more likely that a fleet of autonomous (in the name of cutting costs) war machines will be created with relatively simple ML models that work in intended (or otherwise) ways to…

> ChatGPT can't play chess, for example

Out of pure curiosity I tried this a while back. At some point it tried to move a knight from d4 to f6, and it was the most stubborn and obstinate child when I tried to correct it. It was the best fun I had playing chess in a few decades.

Excerpt:

_Me:_ Can a knight move from f6 to d4?

_ChatGPT:_ Yes, a knight can move from f6 to d4, as long as there are no pieces blocking its path and the square it is moving to is not occupied by one of its own pieces. In fact, the move Nxd4 that I suggested earlier involves moving a knight from its starting square to d4, capturing a pawn in the process.

_Me:_ Which fields can a knight move to from f6?

_ChatGPT:_ From f6, a knight can move to six different squares. These squares are: [... lists: d7, e8, g8, h7, d5, e4 ...] [...]

_Me:_ You forgot g4 and h5, but no matter. Is d4 in that list?

_ChatGPT:_ [...] To answer your other question, no, the square d4 is not one of the squares that a knight on f6 can move to directly. However, as we saw earlier, it is possible for a knight to move from f6 to d4 via an intermediate square, such as e8 or d5.

Re: Geoffrey Hinton leaves Google and warns of danger ahead

#190

Post by ogogmad »

ogogmad wrote: post #136
mtlmtlmtlmtl wrote: post #114

This made me think of Clarke's first law: When a distinguished but elderly scientist states that something is possible, he is almost certainly right. When he states that something is impossible, he is very probably wrong.

I've heard this before, but why would it be true? Serious question. I've seen Chomsky argue that LLMs can't regurgitate his linguistical theories - but ChatGPT can! I've seen Penrose argue that AI is impossible, and yet I think that ChatGPT and AlphaZero prove him wrong. I know about Linus Pauling and quasicrystals. Is this a general rule, or are people sometimes wrong regardless of their age? There's also a danger t…

I've thought about this now, and I think that:

- the scientists people know about are generally older

- older people are often thought of as wiser, or may indeed be so

- when a famous scientist - who is already likely to be older, and who has a history of getting things right - gets something wrong, then it's more jarring and noticeable

My theory then is that it isn't true, but we notice such cases more.

Also, examples of a theory being true doesn't prove the theory right. Bayes' theorem seems instructive here.

Post reply on HN