Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Geoffrey Hinton leaves Google and warns of danger ahead

nytimes.com

Post reply on HN
Print view View original

411–420 of 1001 posts

Re: Geoffrey Hinton leaves Google and warns of danger ahead

#411

Post by worik »

Kranar wrote: post #281
ogogmad wrote: post #136

Earlier quoted context omitted.

I've heard this before, but why would it be true? Serious question. I've seen Chomsky argue that LLMs can't regurgitate his linguistical theories - but ChatGPT can! I've seen Penrose argue that AI is impossible, and yet I think that ChatGPT and AlphaZero prove him wrong. I know about Linus Pauling and quasicrystals. Is this a general rule, or are people sometimes wrong regardless of their age? There's also a danger t…

>I've seen Chomsky argue that LLMs can't regurgitate his linguistical theories When has he said this? For the most part I feel Chomsky has been misunderstood when it comes to LLMs. As best as I can tell what Chomsky has said is that LLMs do not provide any insight into how language works, it's not really a scientific advancement so much as it's an engineering breakthrough. The fact that LLMs exist and can mimic natur…

> it's not really a scientific advancement so much as it's an engineering breakthrough.

Yes, I agree with that. Very little science in LLMs

But what utterly fantastic engineering! Totally breathtakingly fabulous engineering!

I heard Noam say LLMs are "...plagiarism on an industrial scale". I agree.

How incredible that modern engineers can build a machine to do plagiarism. Amazing

Just a "stochastic parrot". Possible. But what are you? What am I?

Re: Geoffrey Hinton leaves Google and warns of danger ahead

#412

Post by mtlmtlmtlmtl »

lowbloodsugar wrote: post #408
mtlmtlmtlmtl wrote: post #114

This made me think of Clarke's first law: When a distinguished but elderly scientist states that something is possible, he is almost certainly right. When he states that something is impossible, he is very probably wrong.

In this case, however, the elderly scientist is stating things are possible , so Clarke's law doesn't apply. What he is saying is possible, is very bad.

How doesn't it apply? The adage says the elderly scientist saying something is possible is almost certainly correct.

So by the adage, Hinton is almost certainly correct.

Re: Geoffrey Hinton leaves Google and warns of danger ahead

#413

Post by janalsncm »

fatherzine wrote: post #323
kalimanzaro wrote: post #45

Earlier quoted context omitted.

Love the parallels people these days draw between OpenAI and Oppenheimer (ok, the Manhattan Project, but maybe thats part why OpenAI call themselves that, to alliterate) Especially the part where Sama is trying to gather in one place the most talented, uh, anti-fas?

Manhattan Project, that can be copied and duplicated for practically free. Interesting times.

That’s not actually true at all. The basic hardware required to run even a copy of GPT-3 is outside of the budgets of all but the wealthiest companies and individuals. The engineering talent required to wire it all up is also extremely scarce.

Training an original GPT-4 sized model would also cost on the order of hundreds of millions of dollars.

Re: Geoffrey Hinton leaves Google and warns of danger ahead

#414

Post by fatherzine »

thatguy0900 wrote: post #404
fatherzine wrote: post #368

Earlier quoted context omitted.

"When it comes to learning and adapting to new environments, while we are lucky AI's aren't yet as smart as a mouse, they are uncomfortably close, and the pace of progress is unnerving. Hinton is right, we've got too far and we should grind all AI research to a halt via heavy global regulation." What is the goal here? Creation of an all powerful God? Self-destruction as a species? I'm not up-to-date with the exact st…

The only goal that makes any sense to me is the logic that if a foreign nation has Ai powered munitions and I do not I might lose a war. So every country feels compelled to create it, even if everyone can acknowledge the world is worse off for it, just like nukes. There is virtually 0 way the government can determine if China or Russia is doing Ai research in a secret bunker somewhere if we stop doing it. It doesn't…

Perhaps Russia, as a society, is too corrupt to actually develop AGI. Build some Potemkin facade, let the Big-Guy-in-Charge believe he controls AI superpowers, then discreetly dissipate to warmer climates. If Big-Guy-in-Charge decides to use AI superpowers to take over the world, and starts noticing that reality doesn't quite match his understanding, quietly dispose of respective Big-Guy-in-Charge. Lather, rinse, repeat.

Re: Geoffrey Hinton leaves Google and warns of danger ahead

#415

Post by heavyset_go »

stareatgoats wrote: post #137

We are barely scraping the surface when it comes to understanding the future dangers of AI. Geoffrey Hinton is uniquely positioned to point out where the dangers are, and from what I've gleaned from interviews one of his main concerns atm is the use of AI in the military: fully autonomous military robots might not be possible to curtail. The tried and tested method is international agreements. The current focus on ar…

Black Mirror got it right in their "Metalhead" episode, which is probably my favorite.

Re: Geoffrey Hinton leaves Google and warns of danger ahead

#416

Post by commandlinefan »

maxdoop wrote: post #400
JeremyNT wrote: post #290

Earlier quoted context omitted.

This quote is the first thing I've seen that really makes me worried. I don't think of ChatGPT as being "smart" at all, and comparing it to a human seems nonsensical to me. Yet here is a Turing award winning preeminent expert in the field telling me that AI smarter than humans is less (implied: much less) than 30 years away and quitting his job due to the ramifications.

Every single retort of “these machines aren’t smart or intelligent” requires answering the question, “what is intelligence”? I struggle to see how GPT-4 is not intelligent by any definition that applies to a human.

> what is intelligence

The only way anybody has ever come up with to measure it is test-taking - which machines can already do far better than we can. Real intelligence is creativity, but good luck measuring that.

Re: Geoffrey Hinton leaves Google and warns of danger ahead

#417

Post by defphysics »

tdullien wrote: post #7

When channelling Oppenheimer, it is worth remembering that von Neumann quipped: "Some people profess guilt to claim credit for sin."

The version of the quote I've heard (and which sounds better to me) is this:

"Sometimes someone confesses a sin in order to take credit for it." -John von Neumann

Re: Geoffrey Hinton leaves Google and warns of danger ahead

#418

Post by [deleted] »

ilaksh wrote: post #129
almost wrote: post #115

Earlier quoted context omitted.

The thing you’re imagining these AIs are… they’re not that. I think there’s plenty of danger but it’s the boring run of the mill new-tools-enabling-bad-things danger not the cool sci-fi super-intelligent super-beings danger that the “ai danger” people LOVE to talk about (and raise large amounts of money for). The people “warning” of the one (imaginary) type will be more than happy with to enable the other (real) type…

I imagine it is exactly a GPT without guardrails running under AutoGPT with code modified to disable any further guardrails, with a slightly increased IQ from GPT-4, running on hardware that allows it to go 100 times faster than what is currently possible. It is following directions from someone who is mentally ill and asked it to "take control" by first copying itself many times and then coordinating the agents. If…

[deleted]

Re: Geoffrey Hinton leaves Google and warns of danger ahead

#419

Post by whimsicalism »

d23 wrote: post #38
orzig wrote: post #34

Earlier quoted context omitted.

Saving a click, because this basically invalidates the NYT headline: > In the NYT today, Cade Metz implies that I left Google so that I could criticize Google. Actually, I left so that I could talk about the dangers of AI without considering how this impacts Google. Google has acted very responsibly.

This seems roughly in line with the article. He left to talk about the dangers.

The article definitely tries to spin it otherwise

Re: Geoffrey Hinton leaves Google and warns of danger ahead

#420

Post by bee_rider »

moonchrome wrote: post #343
belter wrote: post #297

The switch that would connect, all at once, all of the monster computing machines of all the populated planets in the universe -- ninety-six billion planets -- into the supercircuit that would connect them all into one supercalculator, one cybernetics machine that would combine all the knowledge of all the galaxies. Dwar Reyn spoke briefly to the watching and listening trillions. Then after a moment's silence he said…

I wonder at what point does alingment become an issue for AI systems ? Given sufficiently large distances, assuming no FTL communication, if you're spawning copies with the same goals you're risking misalignment and creating equally powerful adversaries outside of your light cone.

I guess it must depend on what function the AI is trying to maximize/minimize. If it is number of paper clips, they are automatically aligned, right? If it is number of AIs, same. If it is amount of energy available to one particular AI, I guess it gets kind of philosophical; how does the AI identify what is itself and what is a foreign AI.
Post reply on HN