Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Rogue superintelligence: Inside the mind of OpenAI's chief scientist

technologyreview.com

Post reply on HN
Print view View original

31–40 of 117 posts

Re: Rogue superintelligence: Inside the mind of OpenAI's chief scientist

#31

Post by [deleted] »

adverbly wrote: post #29
vasco wrote: post #9

These people are too full of themselves. The physicists that invented The Bomb didn't have any special insight into the philosophical and societal implications. It takes a special kind of person to be able to invent something so big it can change the world, but it's about 0% chance that those people can control how the technology then gets used. I wish they'd focus more on the technical advances and less on trying to…

Respectfully, I totally disagree. I am become death, destroyer of worlds. The Nuremberg Trials. And AGI is 10x impact of nuclear. Builders are not cogs, and should not try to be them. We need MORE ethics and good intentions - not less. It is the psychopathic corrupt business rot we should be afraid of - not people actively trying to do good

[deleted]

Re: Rogue superintelligence: Inside the mind of OpenAI's chief scientist

#33

Post by vasco »

jampekka wrote: post #27
vasco wrote: post #9

These people are too full of themselves. The physicists that invented The Bomb didn't have any special insight into the philosophical and societal implications. It takes a special kind of person to be able to invent something so big it can change the world, but it's about 0% chance that those people can control how the technology then gets used. I wish they'd focus more on the technical advances and less on trying to…

Not that people, especially tech oligarchs, aren't full of themselves, but many physicists who were inventing the bomb, most famously Oppenheimer, also laid a lot of the philosophical basis of implications of nuclear weapons. The situation may be a bit different now. The megacorporate world is the biggest threat from these tech developments, and the inventors are getting deeply embedded in this. Sort of similar situa…

They laid them out yet did not have the ability to prevent using it, didn't predict the cold War, and if you asked any of them, the chances the world would exist 10 years after proliferation were basically zero. They were all depressed because of how sure they were that they helped destroy humanity, it was just a matter of time. And here we are. We're talking about the most intelligent humans ever and even they got it wrong.

I'm not saying they can prevent themselves from thinking about the implications, anyone would, but this grandstanding as if nobody else will be able to figure it out or that only them understand the dangers is what is a bit weird.

My main point isn't "don't listen to the inventor", it's more like, "listen to the inventor but don't think that they know the future just because they invented a gadget". These are people that have investment documents saying they don't know what role money will play in post-AGI world. It has the vibes of a cult mixed with role play.

Re: Rogue superintelligence: Inside the mind of OpenAI's chief scientist

#34

Post by koe123 »

tempestn wrote: post #10

The consciousness point is an interesting one. There's probably no way to know, but if biological neural networks manifest consciousness, it certainly seems at the very least plausible that artificial ones would do so as well. The idea of a consciousness that pops in and out of existence seems weird at first, until you realize that ours do that too. When you're "unconscious", the word is literally true. The only thin…

Probably being super naive, but stuff like that always makes me wonder, is there something special about human consciousness, even? Perhaps it's just a sliding scale and we're just seemingly ahead of other specifies. We assert that consciousness exists and distinguishes us, but is that really the case?

Re: Rogue superintelligence: Inside the mind of OpenAI's chief scientist

#35

Post by WillAdams »

vasco wrote: post #33
jampekka wrote: post #27

Earlier quoted context omitted.

Not that people, especially tech oligarchs, aren't full of themselves, but many physicists who were inventing the bomb, most famously Oppenheimer, also laid a lot of the philosophical basis of implications of nuclear weapons. The situation may be a bit different now. The megacorporate world is the biggest threat from these tech developments, and the inventors are getting deeply embedded in this. Sort of similar situa…

They laid them out yet did not have the ability to prevent using it, didn't predict the cold War, and if you asked any of them, the chances the world would exist 10 years after proliferation were basically zero. They were all depressed because of how sure they were that they helped destroy humanity, it was just a matter of time. And here we are. We're talking about the most intelligent humans ever and even they got i…

They all got together and wrote a letter to FDR w/ Einstein as signatory because the implication of not having it was the Axis Powers getting and using a nuclear weapon.

Re: Rogue superintelligence: Inside the mind of OpenAI's chief scientist

#36

Post by minihat »

vasco wrote: post #33
jampekka wrote: post #27

Earlier quoted context omitted.

Not that people, especially tech oligarchs, aren't full of themselves, but many physicists who were inventing the bomb, most famously Oppenheimer, also laid a lot of the philosophical basis of implications of nuclear weapons. The situation may be a bit different now. The megacorporate world is the biggest threat from these tech developments, and the inventors are getting deeply embedded in this. Sort of similar situa…

They laid them out yet did not have the ability to prevent using it, didn't predict the cold War, and if you asked any of them, the chances the world would exist 10 years after proliferation were basically zero. They were all depressed because of how sure they were that they helped destroy humanity, it was just a matter of time. And here we are. We're talking about the most intelligent humans ever and even they got i…

>They were all depressed because of how sure they were that they helped destroy humanity, it was just a matter of time. And here we are.

There's still plenty of anthropocene left for them to be right. At least I hope there is.

Re: Rogue superintelligence: Inside the mind of OpenAI's chief scientist

#37

Post by spigottoday »

tempestn wrote: post #10

The consciousness point is an interesting one. There's probably no way to know, but if biological neural networks manifest consciousness, it certainly seems at the very least plausible that artificial ones would do so as well. The idea of a consciousness that pops in and out of existence seems weird at first, until you realize that ours do that too. When you're "unconscious", the word is literally true. The only thin…

An llm is a database modelled on the concept of a neural (the tissue in our heads) net (because our neural tissue appears to be highly interconected like a net). An llm is a very interesting and complex tool that the creators and many users anthropomorphize this shit out of. Llm's are a spherical cow, a statistical model of a very poorly understood organ and difficult to define process; intelligence. Do we "prompt" our sql databases? When our querry results are incorrect is the database hallucinating? As amazing and usefull as these new ai tools are they are it seems to me they are no more intelligent or conscious than a hammer or guitar or the computer your using right now. I thinks a usefull perspective to apply to ai is competence. To clarify this I will compare an llm to a flea. A flea has a neural network that outperforms any hardware used for ai. Fleas eat, mate, avoid preditors and probably exibite a great deal of complex behavior that we are not aware of. Fleas are highly intelligent competent actors opperationg in a complex environment. Llms are not actors any more than a hammer or a sql database is and only an actor can be competent. I will agree that in most cases llms have more utillity than fleas. Consciousness seems alot more slippery than intelligence which is hard enough by itself.

Re: Rogue superintelligence: Inside the mind of OpenAI's chief scientist

#38

Post by galoisscobi »

I, for one, am glad that Ilya has the reins on OpenAI and that Sam is out of the picture. It does seem that he weighs the ethics of what is being built more heavily than Sam.

I’m also hoping that OpenAI cools down on the regulatory moat they were trying to build as a thinly veiled profit seeking strategy.

Re: Rogue superintelligence: Inside the mind of OpenAI's chief scientist

#39

Post by gibsonf1 »

This is a very delusional idea: "He thinks ChatGPT just might be conscious (if you squint)" It's a technology with literally no intelligence or understanding of the world of any kind. Its just statistics on data. It is as conscious as a calculator.

Re: Rogue superintelligence: Inside the mind of OpenAI's chief scientist

#40

Post by epgui »

orangepurple wrote: post #30
anonzzzies wrote: post #7

I think the immediate problem with AI is none of the sci-fi stuff (which, by the way, has been in sci-fi for many decades and is nothing revolutionary or new; we always expected to go there, just the timelines seem to have compressed, although not really either; most 60-70s scifi set AGI stuff in the begin 90s and begin 00s); I think it's the entire world changing into a helpdesk experience. Everything you try to do,…

Americans would be wise not to give up their guns under any circumstances

Sigh. (From Canada.)
Post reply on HN