Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Geoffrey Hinton leaves Google and warns of danger ahead

nytimes.com

Post reply on HN
Print view View original

271–280 of 1001 posts

Re: Geoffrey Hinton leaves Google and warns of danger ahead

#271

Post by mythrwy »

robotresearcher wrote: post #204
ilaksh wrote: post #107

I used to be fairly unconcerned about AI being dangerous. But part of the Yudkowsky interview on Lex Fridman 's podcast changed my mind. The disconnect for me is that Yudkowsky posits that the AIs will be fully "alive", thinking millions of times faster than humans and that there will be millions of them. This is too big of a speculative leap for me. What I can fairly easily imagine in the next few years with improve…

Why would the AI be running in a loop between queries? It has no work to do, and running costs money.

Same reason we might watch an course video on SQL in the evening after work?

Re: Geoffrey Hinton leaves Google and warns of danger ahead

#272

Post by g9yuayon »

greatpostman wrote: post #98

My honest take is a lot of these famous academics played almost no part in the developments at openai. But they want the limelight. They aren’t as relevant as they want to be. In many cases, they were directly wrong about how ai would develop

In addition to what people clarified in this thread, you probably will be interested in this: Neural network was not a popular research area before 2005. In fact, the AI nuclear winter in the 90s left such a bitter taste that most people thought that NN is a dead end, so much so that Hinton could not even get enough funding for his research. If it were not for Canada's (I forgot the institution's name) miraculous decision to fund Hinton, LeCunn, and Bengio with $10M for 10 years, they probably wouldn't be able to continue their research. I was a CS student in the early 2000s in U of T, a pretty informed one too, yet I did not even know about Hinton's work. At that time, most of the professors who did AI research in U of T were into symbolic reasoning. I still remember I was taking courses like Model Theory and abstract interpretation from one of such professors. Yet Hinton persevered and changed the history.

I don't think Hinton cared about fame as you imagined.

Re: Geoffrey Hinton leaves Google and warns of danger ahead

#274

Post by bluefirebrand »

ttul wrote: post #144
CartyBoston wrote: post #100

Earlier quoted context omitted.

somebody has a no disparage

More like HR said, “Well, there is option A where you leave and are free to do what you wish. And then there is option B (points at bag of cash) where you pretend none of this ever happened…”

HR might as well say:

"It doesn't matter if you take the bags of cash or not, we will do our best to destroy your life if you mess with us after you are gone. The bags of cash are a formality, but you might as well accept them because we have the power to crush you either way"

Re: Geoffrey Hinton leaves Google and warns of danger ahead

#275

Post by hsjqllzlfkf »

O5vYtytb wrote: post #250
stareatgoats wrote: post #137

We are barely scraping the surface when it comes to understanding the future dangers of AI. Geoffrey Hinton is uniquely positioned to point out where the dangers are, and from what I've gleaned from interviews one of his main concerns atm is the use of AI in the military: fully autonomous military robots might not be possible to curtail. The tried and tested method is international agreements. The current focus on ar…

My biggest concern for military use of AI is how incompetent most military contractors are. These huge companies employ an army of not-very-good engineers whose primary purpose seems to be to over-complicate projects. Imagine the same teams that make planes which need to be hard rebooted every few days, now they're making advanced AI to dynamically target and kill people.

It's a bit bizarre that you bring up incompetent engineers but then as an example you mention planes, which are incredibly safe, just because you have to reboot them. It's as if your entire understanding of engineering is writing webapps, and for whom developer-niceties are the main goal, and the value to the user is secondary.

No, planes are a TERRIBLE example of incompetent engineers and your arrogance is breathtaking.

Re: Geoffrey Hinton leaves Google and warns of danger ahead

#276

Post by quonn »

drcode wrote: post #225
ttul wrote: post #164

Earlier quoted context omitted.

Nah, Hinton is already incredibly rich. His first startup was bought by Google for $44M. And Google paid him millions more for a decade. Dr. Hinton is in a rare position of having no reason to work for anyone, not even venture capitalists.

Sam Altman was rich and stated concern for AI x-risk when he started OpenAI

But Hinton was a scientist for decades. Altman was a startup founder. All things being equal, I would assume Hinton is less motivated by money.

Re: Geoffrey Hinton leaves Google and warns of danger ahead

#277

Post by king_magic »

ilaksh wrote: post #107

I used to be fairly unconcerned about AI being dangerous. But part of the Yudkowsky interview on Lex Fridman 's podcast changed my mind. The disconnect for me is that Yudkowsky posits that the AIs will be fully "alive", thinking millions of times faster than humans and that there will be millions of them. This is too big of a speculative leap for me. What I can fairly easily imagine in the next few years with improve…

It wasn't on Lex Friedman's podcast, but on another recent podcast that Yudkowsky said something that has been haunting me:

> but what is the space over which you are unsure?

We have no idea what the mind space of AGI / ASI will be like. I don't particularly want to find out.

Re: Geoffrey Hinton leaves Google and warns of danger ahead

#278

Post by ren_engineer »

stareatgoats wrote: post #137

We are barely scraping the surface when it comes to understanding the future dangers of AI. Geoffrey Hinton is uniquely positioned to point out where the dangers are, and from what I've gleaned from interviews one of his main concerns atm is the use of AI in the military: fully autonomous military robots might not be possible to curtail. The tried and tested method is international agreements. The current focus on ar…

Military application of AI drones isn't even the worst possible use, it's nations using them to completely subjugate their own population(although the same tech could be used against non-peer nations). Combination of things like Gorgon Stare to direct smaller AI controlled drones like what they are using in Ukraine would be a police state nightmare.

https://en.wikipedia.org/wiki/Gorgon_Stare

https://longreads.com/2019/06/21/nothing-kept-me-up-at-night...

they can surveil an entire city in real-time with this and track where everybody is and who they are meeting with. No form of protest or movement against the government will be possible if it's scaled up

Re: Geoffrey Hinton leaves Google and warns of danger ahead

#279

Post by munificent »

non_sequitur wrote: post #243

This was his Tweet from several weeks ago, which I thought was insightful, both from a technical as well as socieconomic perspective when you think about data usage etc in these models - "Caterpillars extract nutrients which are then converted into butterflies. People have extracted billions of nuggets of understanding and GPT-4 is humanity's butterfly." Did he see enough in the past 6 weeks that made him change his…

Note that in that analogy, the caterpillar is dissolved during the process.

Re: Geoffrey Hinton leaves Google and warns of danger ahead

#280

Post by ethbr0 »

dmix wrote: post #258
O5vYtytb wrote: post #250

Earlier quoted context omitted.

My biggest concern for military use of AI is how incompetent most military contractors are. These huge companies employ an army of not-very-good engineers whose primary purpose seems to be to over-complicate projects. Imagine the same teams that make planes which need to be hard rebooted every few days, now they're making advanced AI to dynamically target and kill people.

> now they're making advanced AI to dynamically target and kill people. I'm curious, which projects are working on this? Any ones that we know about? In terms of production weapons I know the newer Switchblades, Excalibur, and even Javelin do some fancy automated targeting before the terminal phase but that's not really AI beyond image recognition and the targets/specific destination are pre-selected. I'm curious wha…

Air defense has had automatic targeting since the 1980s(?) due to the required reaction times.

As for autonomous target/kill programs, that'd be the USAF's Collaborative Combat Aircraft (CCA) program (under NGAD).

It sounds like they're keeping human-in-the-loop for now by targeting a 1:many manned_stealth:drone ratio, but that will be subject to erosion as autonomous capabilities improve.

And the very nature of the mission requires autonomy -- drones penetrating deep into sophisticated air defense systems using stealth and emissions control (so only line-of-sight communication links). Nobody is going to be remotely piloting these things.

Post reply on HN