Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Geoffrey Hinton leaves Google and warns of danger ahead

nytimes.com

Post reply on HN
Print view View original

621–630 of 1001 posts

Re: Geoffrey Hinton leaves Google and warns of danger ahead

#621

Post by zeroxp »

maxdoop wrote: post #400
JeremyNT wrote: post #290

Earlier quoted context omitted.

This quote is the first thing I've seen that really makes me worried. I don't think of ChatGPT as being "smart" at all, and comparing it to a human seems nonsensical to me. Yet here is a Turing award winning preeminent expert in the field telling me that AI smarter than humans is less (implied: much less) than 30 years away and quitting his job due to the ramifications.

Every single retort of “these machines aren’t smart or intelligent” requires answering the question, “what is intelligence”? I struggle to see how GPT-4 is not intelligent by any definition that applies to a human.

Most arguments I've had about this take on a totally different tone when you ask the person if they believe there is more to human consciousness than what is inside the brain. I.e, is there some spiritual element animating our consciousness.

Often, people say yes. Those people almost universally cannot be convinced that a machine is intelligent. But, if they agree the brain is an organ, its not hard to convince them that the functions of that organ can be simulated, like any other.

Re: Geoffrey Hinton leaves Google and warns of danger ahead

#622

Post by cjbprime »

skepticATX wrote: post #229
drcode wrote: post #223

Earlier quoted context omitted.

I think GPT4 can converse on any subject at all as well as a (let's say) 80 IQ human. On some subjects it can converse much better. That feels fundamentally different than a calculator.

GPT-4 is absolutely more generally knowledgeable than any individual person. Individual humans can still easily beat it when it comes to knowledge of individual subjects. Let’s not conflate knowledge with intelligence though. GPT-4 simply isn’t intelligent.

> Individual humans can still easily beat it when it comes to knowledge of individual subjects.

What does a phrase like "GPT-4 scores 90th percentile on the Uniform Bar Exam" mean to you, regarding whether humans can easily surpass its knowledge and reasoning?

https://www.forbes.com/sites/johnkoetsier/2023/03/14/gpt-4-b...

Re: Geoffrey Hinton leaves Google and warns of danger ahead

#623

Post by ya3r »

danShumway wrote: post #525

Another article about fears of AGI. As a reminder, there is not a single LLM on the market today that is not vulnerable to prompt injection, and nobody has demonstrated a fully reliable method to guard against it. And by and large, companies don't really seem to care. Google recently launched a cloud offering that uses a LLM to analyze untrusted code. It's vulnerable to prompt injection through that code. Microsoft B…

> "As a reminder, there is not a single LLM on the market today that is not vulnerable to prompt injection ... And by and large, companies don't really seem to care."

So far from the truth. I know that there are entire teams that specifically work on prompt injection prevention using various techniques inside companies like Microsoft and Google. Companies do care a lot.

Re: Geoffrey Hinton leaves Google and warns of danger ahead

#625

Post by int_19h »

mythrwy wrote: post #271
robotresearcher wrote: post #204

Earlier quoted context omitted.

Why would the AI be running in a loop between queries? It has no work to do, and running costs money.

Same reason we might watch an course video on SQL in the evening after work?

But in this case the owner of the AI decides whether it is running or not, not the AI itself. Why would the owner give it "idle time"?

Re: Geoffrey Hinton leaves Google and warns of danger ahead

#626

Post by int_19h »

NoMoreNicksLeft wrote: post #134
ilaksh wrote: post #107

I used to be fairly unconcerned about AI being dangerous. But part of the Yudkowsky interview on Lex Fridman 's podcast changed my mind. The disconnect for me is that Yudkowsky posits that the AIs will be fully "alive", thinking millions of times faster than humans and that there will be millions of them. This is too big of a speculative leap for me. What I can fairly easily imagine in the next few years with improve…

It is absurd to think of these systems having reproductive instincts. It is so much more absurd to think that they would have these reproductive instincts not by design, but that it's some principle of intelligence itself. Natural intelligences have reproductive instincts because any organism that didn't have them built in within the first few hundred million years have no descendants for you to gawk at as they casua…

Given that we train LLMs on massive amounts of text produced by our own civilization - you know, the one that is to a large extent driven by the innate human desire to reproduce - I would find it more surprising if they did not acquire such an "instinct", regardless of how pointless it might seem.

Re: Geoffrey Hinton leaves Google and warns of danger ahead

#627

Post by danShumway »

ya3r wrote: post #623
danShumway wrote: post #525

Another article about fears of AGI. As a reminder, there is not a single LLM on the market today that is not vulnerable to prompt injection, and nobody has demonstrated a fully reliable method to guard against it. And by and large, companies don't really seem to care. Google recently launched a cloud offering that uses a LLM to analyze untrusted code. It's vulnerable to prompt injection through that code. Microsoft B…

> "As a reminder, there is not a single LLM on the market today that is not vulnerable to prompt injection ... And by and large, companies don't really seem to care." So far from the truth. I know that there are entire teams that specifically work on prompt injection prevention using various techniques inside companies like Microsoft and Google. Companies do care a lot.

They don't care enough to delay the product launches.

There were teams working on Bing search that probably cared a lot about it going off the rails. But the company didn't, it launched anyway even with internal knowledge of its failings.

See also the red flags raised at Google about Bard.

I don't buy this. Companies can demonstrate they care through their choices. Not just by paying an internal team to hopelessly try to solve the problem while their PR and product teams run full speed ahead.

It is a choice for OpenAI to run forward with 3rd-party plugin support while they still don't have an answer to this problem. That choice demonstrates something about the company's values.

Re: Geoffrey Hinton leaves Google and warns of danger ahead

#628

Post by int_19h »

williamcotton wrote: post #21
rvz wrote: post #15

Earlier quoted context omitted.

> The first step is state-issued public-key cryptographic identification cards. Governments totally love this antidote. I wonder who could be selling this sort of snake-oil to them whilst also being on the other side selling the poison... ...No-one else but Sam Altman's World Coin scam. [0] [0] https://worldcoin.org/blog/engineering/humanness-in-the-age-...

I make no case for requiring such identification, rather that it be optional, much like how the post office is optional and FedEx is still allowed to operate!

I think your opt-in approach sounds fine in theory, and I can certainly see many good uses for a reliable proof of identity like that.

But, at the same time, given the history of human governance, I am extremely skeptical that such a scheme would not be co-opted for tracking and surveillance of various outgroups almost immediately, and become mandatory once its utility as such is fully realized.

Re: Geoffrey Hinton leaves Google and warns of danger ahead

#629

Post by nico »

uptownfunk wrote: post #610
nico wrote: post #607

It’s not AI. It’s us. We can choose to make it more equal. We can choose to even things, and to work less. It’s us using the AI to do things. Let’s stop pretending like our hands are tied. We can build a better world if we want to. Don’t give me excuses about how everyone else will do something so then you have to do the same or react in a certain way. Take responsibility for what you can do. If you are in a position…

It sounds nice however the current incentive structures dictated by capitalism make this more a utopian possibility than a realistic one unfortunately

Own up to what you can do.

What are your own incentive structures?

If you won’t do anything, then just be aware that it is your own choice.

Don’t blame incentive structures.

If you are aware of the “incentive structures”, then work around them for what you believe.

Pretty sure that no adverse incentive structures are keeping you from surviving.

So why should they keep you from doing what you believe should be done?

Re: Geoffrey Hinton leaves Google and warns of danger ahead

#630

Post by elevaet »

rossjudson wrote: post #624

Everybody knows procedurally generated game worlds are crap/uninteresting. An infinite supply of variations, where the value of those variations approaches zero. We're headed into a world of procedurally generated culture.

I think ML-generation is in a different class than procedural generation. Sure, technically it's procedural underneath it all, but in practice, this is a different category, and I think the products of ML might end up being more compelling than the procedurally generated game worlds you're talking about.

Take Midjourney for example - the quality, diversity, creativity of the images is subjectively (to me anyways) better than any traditional "procedural" art. When ML starts being able to put whole compelling worlds together... what is that going to be like?

Anyways, your point about infinite supply driving value to approach zero is certainly one thing we can expect.

Post reply on HN