Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Geoffrey Hinton leaves Google and warns of danger ahead

nytimes.com

Post reply on HN
Print view View original

631–640 of 1001 posts

Re: Geoffrey Hinton leaves Google and warns of danger ahead

#631

Post by dclowd9901 »

danShumway wrote: post #525

Another article about fears of AGI. As a reminder, there is not a single LLM on the market today that is not vulnerable to prompt injection, and nobody has demonstrated a fully reliable method to guard against it. And by and large, companies don't really seem to care. Google recently launched a cloud offering that uses a LLM to analyze untrusted code. It's vulnerable to prompt injection through that code. Microsoft B…

Your average reader cannot (and will not) delineate between AGI and an LLM -- I think your concerns are misdirected. If the average person hears "Google AI person left Google to talk freely about the dangers of AI", they're thinking about ChatGPT.

Re: Geoffrey Hinton leaves Google and warns of danger ahead

#632

Post by chii »

godshatter wrote: post #169
ilaksh wrote: post #107

I used to be fairly unconcerned about AI being dangerous. But part of the Yudkowsky interview on Lex Fridman 's podcast changed my mind. The disconnect for me is that Yudkowsky posits that the AIs will be fully "alive", thinking millions of times faster than humans and that there will be millions of them. This is too big of a speculative leap for me. What I can fairly easily imagine in the next few years with improve…

> Take the perspective of one of these things. You think 100 times faster than a person. That means that if it takes 30 seconds for a user to respond or to give you your next instruction, you are waiting 3000 seconds in your loop. For 50 minutes. These things don't have a "perspective". They simply guess based on a lot of statistics from a large language data source what they should say next. They are not going to st…

> create real AI

nobody knows what or how intelligence is actually "implemented" in humans.

There's no need to know how the innards of these large models _actually_ work, if their behaviour is consistent with intelligence.

Re: Geoffrey Hinton leaves Google and warns of danger ahead

#633

Post by bbor »

thinkingemote wrote: post #426
nologic01 wrote: post #4

> the average person will not be able to know what is true anymore We barely held things together as society without AI unleashing cognitive noise at industrial scale. Somehow we must find ways to re-channel the potential of digital technology for the betterment of society, not its annihilation.

There's an argument that people generally do not want the truth and that AI will never be allowed to tell it. An optimist could view this as ensuring AI will be safe forever or pessimistically they might see it as AI never being authoritative ever. One example of truth would be the topic of biological sex another about politics or economics or racism. Imagine releasing an AI that told the actual truth. It's impossibl…

Ok

A) It's not possible to build a machine that knows the absolute truth, that's fundamentally impossible; induction is impossible, and there are hordes (well... dozens?) of Epistemologists concerned with finding and defining the very small corners of knowledge that we _can_ be certain about, such as "a triangle has three sides" or "an orange is an orange".

B) If that angered/interested you, you should look into Standpoint Theory! It's a very interesting discussion on how humans operate with significant bias at all levels of thought, and pretending otherwise is a disservice to science. And this is using "bias" in a very broader sense.

B) Are we allowed to berate/report/etc. ""race realists"" on HN? I know the rules are big on positive interaction, so I hope it's not out of ine to say that's some obvious scared-white-man bullshit that has no place in this community.

Re: Geoffrey Hinton leaves Google and warns of danger ahead

#634

Post by sanderjd »

danShumway wrote: post #525

Another article about fears of AGI. As a reminder, there is not a single LLM on the market today that is not vulnerable to prompt injection, and nobody has demonstrated a fully reliable method to guard against it. And by and large, companies don't really seem to care. Google recently launched a cloud offering that uses a LLM to analyze untrusted code. It's vulnerable to prompt injection through that code. Microsoft B…

I think it's going to be hard to get people to care about this until you can point to a concrete attack.

Like you said, Google and Bing have running high visibility widely used services that are vulnerable to this problem for awhile now. What attacks have there been in that time?

Re: Geoffrey Hinton leaves Google and warns of danger ahead

#635

Post by noduerme »

danShumway wrote: post #608
noduerme wrote: post #570

Earlier quoted context omitted.

>> if you're genuinely losing sleep about GPT-4 becoming a general agent that does every job I guess I'm one of those people, because I'm not convinced that GPT-3.5 didn't do some heavy lifting in training GPT-4... that is the take-off. The fact that there are still some data scientists or coders or "ethics committees" in the loop manifestly is not preventing AI from accelerating its own development. Unless you belie…

> In any event, you make a good case that can be extended: If the companies throwing endless processing at LLMs can't even conceive of a way to prioritize injection threats thought up by humans, how would they even notice LLMs injecting each other or themselves for nefarious purposes? I would love to read press articles that dove into this. There's a way of talking about more future-facing concerns that doesn't give…

Dan, from a purely realpolitik standpoint, these companies don't even want to be implicated as having control of their own software now. Any attempt to do so would hinder the mission. The question is... is it even their mission anymore? From a certain perspective, they might already be buying up hardware for an AI who is essentially demanding it from them. In that case the takeoff is happening right now. Dismissing basic security protocols should be totally anathema to devs in the 2020s. That's not "moving fast and breaking things"... a slightly paranoid mind could see it as something else.

I think that they (OpenAI, Alphabet) think that the ladder can be climbed by leveraging GPT and LLMs until they have AGI. I think they think whoever gets AGI first takes all the chips off the table and rules the world forever forward. While these endless, idiotic debates happen as to whether GPT is or ever could be "alive" or whatever, they're actively employing it to build the one ring that'll rule them all. And I think the LLM model structure is capable of at least multiplying human intelligence enough to accomplish that over a couple more iterations, if not capable of conceiving the exact problems for itself yet.

There's also no real economic incentive to develop AGI that benefits everyone... Sam Altman's strangely evasive remarks to the contrary. There is every incentive to develop one for dominance. The most powerful extant tool to develop AGI right now is GPT-4.

Re: Geoffrey Hinton leaves Google and warns of danger ahead

#637

Post by intalentive »

nathan_gold wrote: post #430
intalentive wrote: post #337

We still don’t have Rosie the Robot. When it comes to learning and adapting to new environments, we don’t even have AI as smart as a mouse. LeCun is right, there is still a long way to go.

It’s very clear you are not a user of GPT4.

Can it fetch me a beer from the fridge?

Re: Geoffrey Hinton leaves Google and warns of danger ahead

#638

Post by fuzzfactor »

meh8881 wrote: post #128
greatpostman wrote: post #98

My honest take is a lot of these famous academics played almost no part in the developments at openai. But they want the limelight. They aren’t as relevant as they want to be. In many cases, they were directly wrong about how ai would develop

Regardless of incentives, I don’t see any particular reason to think he has a more informed view than other experts on the trajectory of AI. He’s made several incorrect bets (capsule networks). I’m sure he’s smart and all. His contributions were valuable. But he’s not special in this particular moment.

There's something about being first that gives a pioneer a great head start that can't be matched when it comes to considering the implications of their groundbreaking work.

Even if they're too busy doing the work, they're still thinking about what it would be like if it performed successfully, and it does seem to always take more retrospection before a leader can fully raise their head and more carefully consider unintended consequences.

Early success can give the impression that future efforts have difficulty being as meaningful, but also realistically after that the successful individual does not need to struggle to prove themself any more the way the less-accomplished would be expected to do.

Then there's seniority itself, and maturity levels that can not be gained any other way.

Beyond that when retirement is within easy reach you don't really have the same obligation to decorum itself as you would earlier, in order to actually maintain the same desired level of decorum.

Dr. Hinton seems to do a pretty good job of comparing himself to Oppenheimer.

I don't see how anyone else can question his standing more seriously than that.

Re: Geoffrey Hinton leaves Google and warns of danger ahead

#639

Post by thundergolfer »

areyousure wrote: post #235
edgefield wrote: post #132

Earlier quoted context omitted.

It sounds like you’re biased against academics. Not only did Hinton develop some of the fundamental ideas behind AI (winning the Turing award) but also one of his PhD students is now the CTO at OpenAI.

In case anyone is curious, this appears to refer to https://en.wikipedia.org/wiki/Ilya_Sutskever who was a PhD student of Geoffrey Hinton's and is now Chief Scientist at OpenAI. The CTO at OpenAI is https://en.wikipedia.org/wiki/Mira_Murati who does not have a PhD.

Wow the CTO of OpenAi seems to have ~1 yr of hands on engineering experience, followed by years of product and people management, That’s unexpected. I thought the CTO was Brockman.
Post reply on HN