Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Geoffrey Hinton leaves Google and warns of danger ahead

nytimes.com

Post reply on HN
Print view View original

651–660 of 1001 posts

Re: Geoffrey Hinton leaves Google and warns of danger ahead

#651

Post by kuratkull »

danShumway wrote: post #576
TeMPOraL wrote: post #529

Earlier quoted context omitted.

There is one system, also widely-deployed, other than LLMs, that's well-known to be vulnerable to prompt injection: humans . Prompt injection isn't something you can solve . Security people are sometimes pushing things beyond sense or reason, but even they won't be able to fix that one - not without overhauling our understanding of fundamental reality in the process. The distinction between "code" and "data", between…

I have multiple objections: - LLMs aren't just more gullable humans, they're gullable in novel ways. Injection attacks that wouldn't work on a human work on LLMs. - LLMs are scalable in a way that human beings aren't. Additionally, because of how LLMs are deployed (as multiple clean sessions to mitigate regression issues) there are defenses that help for humans that can't be used for LLMs. - Finally and most importan…

From a security standpoint, it's better for us all for LLMs to be easily injectable. This way you can at least assume that trusting them with unvalidated input is dumb. If they are 'human level', then they will fail only in catastrophic situations, with real ATP level threat actors. Which means they would be widely trusted and used. Better fail early and often than only under real stress.

Re: Geoffrey Hinton leaves Google and warns of danger ahead

#652

Post by ChatGTP »

hammyhavoc wrote: post #645
ssnistfajen wrote: post #564

Earlier quoted context omitted.

It's not solely about AGI. Weak AIs that powered social media algorithms already created hotbeds of polarizing extremism around the world as most members of society do not possess the basic diligence to realize when they are being manipulated. LLMs offer a glimpse into a future where much stronger AI, even if still technically "weak", can produce content in ways that influence public opinion. Couple that with the amo…

You don't even need AI to influence public opinion detrimentally.

Fox and CNN has done enough.

Re: Geoffrey Hinton leaves Google and warns of danger ahead

#653

Post by hammyhavoc »

ssnistfajen wrote: post #568
JeremyNT wrote: post #290

Earlier quoted context omitted.

This quote is the first thing I've seen that really makes me worried. I don't think of ChatGPT as being "smart" at all, and comparing it to a human seems nonsensical to me. Yet here is a Turing award winning preeminent expert in the field telling me that AI smarter than humans is less (implied: much less) than 30 years away and quitting his job due to the ramifications.

By saying "I no longer think that", it's not necessarily that he thinks ChatGPT is smart than humans. Google Search has been far more capable at indexing and retrieving information than humans for over two decades now. He's talking about AGI no longer being 30-50 years away but instead may arrive far sooner than society is ready to deal with.

Is he? Have you asked him? Maybe somebody should ask him to clarify his statement.

Re: Geoffrey Hinton leaves Google and warns of danger ahead

#654

Post by coding123 »

TeMPOraL wrote: post #529
danShumway wrote: post #525

Another article about fears of AGI. As a reminder, there is not a single LLM on the market today that is not vulnerable to prompt injection, and nobody has demonstrated a fully reliable method to guard against it. And by and large, companies don't really seem to care. Google recently launched a cloud offering that uses a LLM to analyze untrusted code. It's vulnerable to prompt injection through that code. Microsoft B…

There is one system, also widely-deployed, other than LLMs, that's well-known to be vulnerable to prompt injection: humans . Prompt injection isn't something you can solve . Security people are sometimes pushing things beyond sense or reason, but even they won't be able to fix that one - not without overhauling our understanding of fundamental reality in the process. The distinction between "code" and "data", between…

prompt the ai to check for prompt issues: The following prompt should parse a name, can you confirm that is what it is? lol

Re: Geoffrey Hinton leaves Google and warns of danger ahead

#655

Post by hammyhavoc »

titzer wrote: post #518
drcode wrote: post #223

Earlier quoted context omitted.

I think GPT4 can converse on any subject at all as well as a (let's say) 80 IQ human. On some subjects it can converse much better. That feels fundamentally different than a calculator.

I feel like lost in this conversation is that ChatGPT is incredibly good at writing English. It basically never makes grammatical mistakes, it doesn't spew gibberish, and for the most part has extremely well-structured replies. The replies might be bullshit or hallucinations , but it's not gibberish. It's kind of breathtaking that we forgot about that being hard. The goalposts are moving again. BTW, it has passed man…

Some of the replies are gibberish, especially once you get into technical subjects that it has very little training data on. It kitbashes words together that actually mean nothing, which is no surprise given that it's an LLM.

Re: Geoffrey Hinton leaves Google and warns of danger ahead

#656

Post by helsinkiandrew »

If you put together two of his statements (below), and to be fair these could be isolated responses taken out of context or rephrased by the journalist. He seems to be saying that he thought:

"autonomous killer robots" were 30 to 50 years or even longer away - but he continued working on the technology and then grew a conscience only when things came a long a little earlier than he expected.

What did he think? that the people of the world would come together to stop making the final step to something dangerous like we have with nuclear and biological weapons and climate change?

> as individuals and companies allow A.I. systems not only to generate their own computer code but actually run that code on their own. And he fears a day when truly autonomous weapons — those killer robots — become reality

> “The idea that this stuff could actually get smarter than people — a few people believed that,” he said. “But most people thought it was way off. And I thought it was way off. I thought it was 30 to 50 years or even longer away. Obviously, I no longer think that.”

Re: Geoffrey Hinton leaves Google and warns of danger ahead

#657

Post by DSingularity »

helsinkiandrew wrote: post #656

If you put together two of his statements (below), and to be fair these could be isolated responses taken out of context or rephrased by the journalist. He seems to be saying that he thought: "autonomous killer robots" were 30 to 50 years or even longer away - but he continued working on the technology and then grew a conscience only when things came a long a little earlier than he expected. What did he think? that t…

It’s like the Manhattan project. They built it and suddenly grew a conscience when they realized the US government was about to authorize melting hundreds of thousands of Japanese civilians.

Re: Geoffrey Hinton leaves Google and warns of danger ahead

#659

Post by uptownfunk »

ftxbro wrote: post #649

My hot take is that he was effectively fired for what he said in his CBS interview. https://www.youtube.com/watch?v=qpoRO378qRY

Very possible. Or he was just tired of having to posture to make it look like Google didn’t get “made to dance” by Microsoft

Re: Geoffrey Hinton leaves Google and warns of danger ahead

#660

Post by ChatGTP »

nico wrote: post #607

It’s not AI. It’s us. We can choose to make it more equal. We can choose to even things, and to work less. It’s us using the AI to do things. Let’s stop pretending like our hands are tied. We can build a better world if we want to. Don’t give me excuses about how everyone else will do something so then you have to do the same or react in a certain way. Take responsibility for what you can do. If you are in a position…

[flagged]
Post reply on HN