Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Geoffrey Hinton leaves Google and warns of danger ahead

nytimes.com

Post reply on HN
Print view View original

531–540 of 1001 posts

Re: Geoffrey Hinton leaves Google and warns of danger ahead

#531

Post by m3kw9 »

maxdoop wrote: post #400
JeremyNT wrote: post #290

Earlier quoted context omitted.

This quote is the first thing I've seen that really makes me worried. I don't think of ChatGPT as being "smart" at all, and comparing it to a human seems nonsensical to me. Yet here is a Turing award winning preeminent expert in the field telling me that AI smarter than humans is less (implied: much less) than 30 years away and quitting his job due to the ramifications.

Every single retort of “these machines aren’t smart or intelligent” requires answering the question, “what is intelligence”? I struggle to see how GPT-4 is not intelligent by any definition that applies to a human.

It’s intelligent but very short sighted, it can’t plan far beyond and really self generate independently beyond the initial few prompts

Re: Geoffrey Hinton leaves Google and warns of danger ahead

#532

Post by p1necone »

danShumway wrote: post #525

Another article about fears of AGI. As a reminder, there is not a single LLM on the market today that is not vulnerable to prompt injection, and nobody has demonstrated a fully reliable method to guard against it. And by and large, companies don't really seem to care. Google recently launched a cloud offering that uses a LLM to analyze untrusted code. It's vulnerable to prompt injection through that code. Microsoft B…

I think you're vastly overestimating how much people care about security.

Re: Geoffrey Hinton leaves Google and warns of danger ahead

#533

Post by ljw1001 »

TeMPOraL wrote: post #529
danShumway wrote: post #525

Another article about fears of AGI. As a reminder, there is not a single LLM on the market today that is not vulnerable to prompt injection, and nobody has demonstrated a fully reliable method to guard against it. And by and large, companies don't really seem to care. Google recently launched a cloud offering that uses a LLM to analyze untrusted code. It's vulnerable to prompt injection through that code. Microsoft B…

There is one system, also widely-deployed, other than LLMs, that's well-known to be vulnerable to prompt injection: humans . Prompt injection isn't something you can solve . Security people are sometimes pushing things beyond sense or reason, but even they won't be able to fix that one - not without overhauling our understanding of fundamental reality in the process. The distinction between "code" and "data", between…

If you don’t consider the difference in kind between a human vulnerability and an automated vulnerability that derives from the essentially unlimited capacity of the latter to scale, your comment makes a lot of sense. If you do consider that, the argument becomes irrelevant and deeply misleading

Re: Geoffrey Hinton leaves Google and warns of danger ahead

#534

Post by m3kw9 »

nologic01 wrote: post #4

> the average person will not be able to know what is true anymore We barely held things together as society without AI unleashing cognitive noise at industrial scale. Somehow we must find ways to re-channel the potential of digital technology for the betterment of society, not its annihilation.

Which is fine, humans will adapt to this info noise rather than going crazy, Hinton is way underestimating human intelligence

Re: Geoffrey Hinton leaves Google and warns of danger ahead

#535

Post by yeahwhatever10 »

tpowell wrote: post #483
ec664 wrote: post #80

See his response on twitter. https://twitter.com/geoffreyhinton/status/165299357072121037...

Yesterday, I randomly watched his full interview from a month ago with CBS Morning, and found the discussion much more nuanced than today's headlines. https://www.youtube.com/watch?v=qpoRO378qRY&t=16s The next video in my recommendations was more dire, but equally as interesting: https://www.youtube.com/watch?v=xoVJKj8lcNQ&t=2847s

I don't understand the "safety" concerns from the example in the second video.

Re: Geoffrey Hinton leaves Google and warns of danger ahead

#536

Post by UberFly »

revelio wrote: post #145
nologic01 wrote: post #4

> the average person will not be able to know what is true anymore We barely held things together as society without AI unleashing cognitive noise at industrial scale. Somehow we must find ways to re-channel the potential of digital technology for the betterment of society, not its annihilation.

Society will be fine, actually AI will make things much better, just as the internet did. People have been making these kind of extreme predictions for decades and it was always wrong. The only people still upset about better communications tech are the people who pine for the days when all that was expected of respectable people was automatically trusting anyone working for the government, a university or a newspape…

"...freakout about the "dangers" of AI is really just a part of the ongoing freakout over losing control over information flows..."

Not all of the "information flows" you mention are helpful or benevolent. Most will likely be targeted and hyper-focused to manipulate individuals like they are now.

Re: Geoffrey Hinton leaves Google and warns of danger ahead

#538

Post by slashdev »

Eisenstein wrote: post #504
slashdev wrote: post #181

Earlier quoted context omitted.

It’s not the war robots that worry me as much as centralized intelligence with internet connectivity. War robots don’t reproduce, require energy infrastructure, and can be destroyed. While they could run amok, by targeting things they’re not supposed to, they won’t really be intelligent because the problem doesn’t require much intelligence. Now if they’re controlled by a central intelligence that’s a bit scarier.

What if the only way to stop war from occurring is to entirely remove humans from the decision making that would cause it?

You don’t get that with war robots, the decision to go to war or not is still in the hands of the politicians.

But I wonder sometimes what it would take to end war.

Re: Geoffrey Hinton leaves Google and warns of danger ahead

#539

Post by wslh »

4rt wrote: post #522
wslh wrote: post #484

Do you think that this story has some similarities with the movie WarGames (1983) [1] ? I am connecting Geoffrey Hinton with the Stephen Falken character in the movie [2] [1] https://en.wikipedia.org/wiki/WarGames [2] https://war-games.fandom.com/wiki/Stephen_Falken

"Colossus: The Forbin Project" https://www.imdb.com/title/tt0064177/ I prefer this example.

Will see it, one friend told me that is available on Internet.

Without asking for hard spoilers: is there an AI researcher that quits a government agency?

Re: Geoffrey Hinton leaves Google and warns of danger ahead

#540

Post by zmmmmm »

danShumway wrote: post #525

Another article about fears of AGI. As a reminder, there is not a single LLM on the market today that is not vulnerable to prompt injection, and nobody has demonstrated a fully reliable method to guard against it. And by and large, companies don't really seem to care. Google recently launched a cloud offering that uses a LLM to analyze untrusted code. It's vulnerable to prompt injection through that code. Microsoft B…

> that'll only last until it gets someone's bank account emptied or until some enemy combatant uses prompt injection to get a drone to bomb a different target

You're joining dots from LLM's producing text output that humans read to them being linked to autonomously taking actions by themselves. That's a huge leap. I think that's the risk that needs to be focused on, not the general concept of the technology.

And what I see as most critical is to establish legal frameworks around liability for anybody who does that being correctly associated. What we can't have is AI being linked to real world harm and then nobody being accountable because "the AI did it". We already have this with traditional computing where you phone up a company with a reasonable request and what would otherwise be an outrageous refusal turns into apparently acceptable outcome because "computer says no". Similarly with people's Google or Meta accounts being auto-banned by bots and their online lives destroyed while their desperate pleas for help are auto-replied to with no way to reach a human.

But it is all a separate problem in my eyes - and not actually something specific to AI.

Post reply on HN