Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Geoffrey Hinton leaves Google and warns of danger ahead

nytimes.com

Post reply on HN
Print view View original

311–320 of 1001 posts

Re: Geoffrey Hinton leaves Google and warns of danger ahead

#311

Post by sib »

ogogmad wrote: post #136
mtlmtlmtlmtl wrote: post #114

This made me think of Clarke's first law: When a distinguished but elderly scientist states that something is possible, he is almost certainly right. When he states that something is impossible, he is very probably wrong.

I've heard this before, but why would it be true? Serious question. I've seen Chomsky argue that LLMs can't regurgitate his linguistical theories - but ChatGPT can! I've seen Penrose argue that AI is impossible, and yet I think that ChatGPT and AlphaZero prove him wrong. I know about Linus Pauling and quasicrystals. Is this a general rule, or are people sometimes wrong regardless of their age? There's also a danger t…

Max Planck said:

    A new scientific truth does not triumph by convincing its opponents and making them see the light, but rather because its opponents eventually die and a new generation grows up that is familiar with it ...

    An important scientific innovation rarely makes its way by gradually winning over and converting its opponents: it rarely happens that Saul becomes Paul. What does happen is that its opponents gradually die out, and that the growing generation is familiarized with the ideas from the beginning: another instance of the fact that the future lies with the youth.
Seems largely in line with Clarke's comment.

Re: Geoffrey Hinton leaves Google and warns of danger ahead

#312

Post by ikekkdcjkfke »

dmix wrote: post #258
O5vYtytb wrote: post #250

Earlier quoted context omitted.

My biggest concern for military use of AI is how incompetent most military contractors are. These huge companies employ an army of not-very-good engineers whose primary purpose seems to be to over-complicate projects. Imagine the same teams that make planes which need to be hard rebooted every few days, now they're making advanced AI to dynamically target and kill people.

> now they're making advanced AI to dynamically target and kill people. I'm curious, which projects are working on this? Any ones that we know about? In terms of production weapons I know the newer Switchblades, Excalibur, and even Javelin do some fancy automated targeting before the terminal phase but that's not really AI beyond image recognition and the targets/specific destination are pre-selected. I'm curious wha…

You can bet they have automatic target tracking and ballistic aiming onto those in the new abrams tank, but sending the kill command will most likely be done by operators. But it's just a small step from that to putting it on autopilot

Re: Geoffrey Hinton leaves Google and warns of danger ahead

#313

Post by ilaksh »

loudmax wrote: post #177
ilaksh wrote: post #107

I used to be fairly unconcerned about AI being dangerous. But part of the Yudkowsky interview on Lex Fridman 's podcast changed my mind. The disconnect for me is that Yudkowsky posits that the AIs will be fully "alive", thinking millions of times faster than humans and that there will be millions of them. This is too big of a speculative leap for me. What I can fairly easily imagine in the next few years with improve…

> So eventually you get agents that have self-preservation and reproductive instincts. I'm not sure that's a given. Artificial Intelligence as it currently exists, doesn't have any volition. AI doesn't have desire or fear, the way natural biological intelligence does. So you may be able to build a directive for self-preservation or reproduction into an artificial intelligence, but there's no particular reason to expe…

I didn't mean they get any characteristic by accident or spontaneously or something. I think that's ridiculous and people talking about that are confusing the issues here.

I liked Avogadro Corp. Good book.

It's true that people will be directing these AIs initially but some people are already giving them incredibly broad goals that could be interpreted as "take over". And there are quite a few developers earnestly working on emulating those lifelike characteristics. So even though they are not going to "emerge" science fiction style, self-preservation and reproductive goals are explicitly being built into these systems by some developers.

Re: Geoffrey Hinton leaves Google and warns of danger ahead

#314

Post by ska »

vbezhenar wrote: post #54
archerx wrote: post #36

There is part of me that thinks that this A.I. fear-mongering is some kind of tactic by Google to get everybody to pause training their A.I.s so they can secretly catch up in the background. If I was to do some quick game theory in my mind this would be the result. Imagine being Google, leading the way in A.I. for years, create the frameworks (tensorflow), create custom hardware for A.I. (TPUs), fund a ton of researc…

The fear is from people who can extrapolate. Who can remember state of AI 20/10/5 years ago. And compare it to 2023. Whether that extrapolation makes sense, nobody knows. But fear is understandable.

> The fear is from people who can extrapolate.

This isn't really true. There isn't consensus among people who have the history and background, but the "it's going to change everything" and especially "we're all screwed" make for better copy so they are getting a lot of media play right now.

Re: Geoffrey Hinton leaves Google and warns of danger ahead

#315

Post by chrsjxn »

neatze wrote: post #99

“The idea that this stuff could actually get smarter than people — a few people believed that,” said Hinton to the NYT. “But most people thought it was way off. And I thought it was way off. I thought it was 30 to 50 years or even longer away. Obviously, I no longer think that.” Calculators are smarter then humans in calculating, what does he mean by that?

That statement seems like such science fiction that it's kind of baffling an AI expert said it.

What does it even mean for the AI to be smarter than people? I certainly can't see a way for LLMs to generate "smarter" text than what's in their training data.

And even the best case interactions I've seen online still rely on human intelligence to guide the AI to good outcomes instead of bad ones.

Writing is a harder task to automate than calculation, but the calculator example seems pretty apt.

Re: Geoffrey Hinton leaves Google and warns of danger ahead

#316

Post by wellthisisgreat »

greatpostman wrote: post #98

My honest take is a lot of these famous academics played almost no part in the developments at openai. But they want the limelight. They aren’t as relevant as they want to be. In many cases, they were directly wrong about how ai would develop

lol literally chief scientist of openai is GH student

Re: Geoffrey Hinton leaves Google and warns of danger ahead

#317

Post by calf »

g9yuayon wrote: post #272
greatpostman wrote: post #98

My honest take is a lot of these famous academics played almost no part in the developments at openai. But they want the limelight. They aren’t as relevant as they want to be. In many cases, they were directly wrong about how ai would develop

In addition to what people clarified in this thread, you probably will be interested in this: Neural network was not a popular research area before 2005. In fact, the AI nuclear winter in the 90s left such a bitter taste that most people thought that NN is a dead end, so much so that Hinton could not even get enough funding for his research. If it were not for Canada's (I forgot the institution's name) miraculous dec…

I remember in 2010 a postdoc came to teach a course on model checking and the classroom was just packed with CS students.

I never took it but it will be interesting to see what kind of synthesis between traditional logic and neural network paradigms can be achieved.

Re: Geoffrey Hinton leaves Google and warns of danger ahead

#318

Post by scarmig »

belter wrote: post #297

The switch that would connect, all at once, all of the monster computing machines of all the populated planets in the universe -- ninety-six billion planets -- into the supercircuit that would connect them all into one supercalculator, one cybernetics machine that would combine all the knowledge of all the galaxies. Dwar Reyn spoke briefly to the watching and listening trillions. Then after a moment's silence he said…

That reminds me of this, more optimistically:

  Matter and energy had ended and with it space and time. Even AC [Automated Computer] existed only for the sake of the one last question that it had never answered from the time a half-drunken computer technician ten trillion years before had asked the question of a computer that was to AC far less than was a man to Man.
  All other questions had been answered, and until this last question was answered also, AC might not release his consciousness.
  All collected data had come to a final end. Nothing was left to be collected.
  But all collected data had yet to be completely correlated and put together in all possible relationships.
  A timeless interval was spent in doing that.
  And it came to pass that AC learned how to reverse the direction of entropy.
  But there was now no man to whom AC might give the answer of the last question. No matter. The answer -- by demonstration -- would take care of that, too.
  For another timeless interval, AC thought how best to do this. Carefully, AC organized the program.
  The consciousness of AC encompassed all of what had once been a Universe and brooded over what was now Chaos. Step by step, it must be done.
  And AC said, "LET THERE BE LIGHT!"
  And there was light --
https://users.ece.cmu.edu/~gamvrosi/thelastq.html

(Interesting, "The Last Question" was published in 1956, two years after "Answer." I wonder if Asimov was influenced by it.)

ETA: ChatGPT says: Isaac Asimov acknowledged the influence of Fredric Brown's "Answer" in his book "Asimov on Science Fiction," where he wrote: "I was also much taken by Fredric Brown's 'Answer,' which appeared in Galaxy Science Fiction in the 1950s."

This is, as far as I can tell, an entirely invented quote. Fiat factum.

Re: Geoffrey Hinton leaves Google and warns of danger ahead

#319

Post by Al-Khwarizmi »

chrsjxn wrote: post #315
neatze wrote: post #99

“The idea that this stuff could actually get smarter than people — a few people believed that,” said Hinton to the NYT. “But most people thought it was way off. And I thought it was way off. I thought it was 30 to 50 years or even longer away. Obviously, I no longer think that.” Calculators are smarter then humans in calculating, what does he mean by that?

That statement seems like such science fiction that it's kind of baffling an AI expert said it. What does it even mean for the AI to be smarter than people? I certainly can't see a way for LLMs to generate "smarter" text than what's in their training data. And even the best case interactions I've seen online still rely on human intelligence to guide the AI to good outcomes instead of bad ones. Writing is a harder tas…

> I certainly can't see a way for LLMs to generate "smarter" text than what's in their training data.

Their training data contains much more knowledge than any single human has ever had, though. If they had equivalent linguistic, understanding and reasoning abilities to a human, but with so much stored knowledge, and considering that they also win in processing speed and never get tired, that would already make them much "smarter" than humans.

Not to mention that LLMs are just the current state of the art. We don't know if there will be another breakthrough which will counter the limitation you are mentioning. We do know that AI breakthroughs are relatively common lately.

Re: Geoffrey Hinton leaves Google and warns of danger ahead

#320

Post by [deleted] »

corbulo wrote: post #303
meroes wrote: post #299

The flip-flopping of AI critics is completely explainable by flip-flopping morals of the architects. > Dr. Hinton said that when people used to ask him how he could work on technology that was potentially dangerous, he would paraphrase Robert Oppenheimer, who led the U.S. effort to build the atomic bomb: “When you see something that is technically sweet, you go ahead and do it.” If anyone outside the core architects…

Would the world be better off without MAD?

[deleted]
Post reply on HN