Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Rogue superintelligence: Inside the mind of OpenAI's chief scientist

technologyreview.com

Post reply on HN
Print view View original

41–50 of 117 posts

Re: Rogue superintelligence: Inside the mind of OpenAI's chief scientist

#41

Post by beowulfey »

vasco wrote: post #33
jampekka wrote: post #27

Earlier quoted context omitted.

Not that people, especially tech oligarchs, aren't full of themselves, but many physicists who were inventing the bomb, most famously Oppenheimer, also laid a lot of the philosophical basis of implications of nuclear weapons. The situation may be a bit different now. The megacorporate world is the biggest threat from these tech developments, and the inventors are getting deeply embedded in this. Sort of similar situa…

They laid them out yet did not have the ability to prevent using it, didn't predict the cold War, and if you asked any of them, the chances the world would exist 10 years after proliferation were basically zero. They were all depressed because of how sure they were that they helped destroy humanity, it was just a matter of time. And here we are. We're talking about the most intelligent humans ever and even they got i…

How can you be sure that it wasn’t their efforts that allowed us to continue existing? Maybe if they hadn’t made the destructive implication of their work clear no one else would have recognized it. None of us can say.

It’s obvious in hindsight, but can we really say with certainty that things would have been exactly the same if the inventors did or said nothing? I’m not willing to take that bet.

To me, effort such as this is always worth it, even if it DOES have no effect, because the chance it can change things for the better is worth it.

Re: Rogue superintelligence: Inside the mind of OpenAI's chief scientist

#42

Post by Cacti »

adverbly wrote: post #29
vasco wrote: post #9

These people are too full of themselves. The physicists that invented The Bomb didn't have any special insight into the philosophical and societal implications. It takes a special kind of person to be able to invent something so big it can change the world, but it's about 0% chance that those people can control how the technology then gets used. I wish they'd focus more on the technical advances and less on trying to…

Respectfully, I totally disagree. I am become death, destroyer of worlds. The Nuremberg Trials. And AGI is 10x impact of nuclear. Builders are not cogs, and should not try to be them. We need MORE ethics and good intentions - not less. It is the psychopathic corrupt business rot we should be afraid of - not people actively trying to do good

you talk about AGI like it is an actual thing and not just some delusional pipe dream that these nerds use to tell themselves they are different from the Valley, they’re not just there for the obscene amount of money, and their shit doesn’t stink.

Re: Rogue superintelligence: Inside the mind of OpenAI's chief scientist

#43

Post by Cacti »

mattigames wrote: post #26
sfn42 wrote: post #22

Earlier quoted context omitted.

Whatever steps they choose not to take, someone else will. And LLMs aren't taking over the world any time in the foreseeable future, they're glorified parrots.

The former president of the US is a glorified parrot, sometimes not even good at that, and still he was able to become president, so I wouldn't hold a defense for it to just that.

what does that have to do with anything?

Re: Rogue superintelligence: Inside the mind of OpenAI's chief scientist

#44

Post by patcon »

galoisscobi wrote: post #38

I, for one, am glad that Ilya has the reins on OpenAI and that Sam is out of the picture. It does seem that he weighs the ethics of what is being built more heavily than Sam. I’m also hoping that OpenAI cools down on the regulatory moat they were trying to build as a thinly veiled profit seeking strategy.

I was speaking with a woman earlier this week who just finished her master's dissertation with a focus on Sam Altman's recent influence campaign, and she was scared of how charming he was. She obviously was trying to maintain distance for impartiality, but his interview style, and the seeming total sincerity of his communication style... It was terrifying to her in how well he could draw everyone in. She was absolutely endeared by him, but aware of how powerful that endearment was, and so scared and worried about his impact.

Re: Rogue superintelligence: Inside the mind of OpenAI's chief scientist

#45

Post by mistermann »

sfn42 wrote: post #25
_ugfj wrote: post #2

> his new priority is to figure out how to stop an artificial superintelligence (a hypothetical future technology he sees coming with the foresight of a true believer) from going rogue. that's cute. What worries me is the here and now leading to a very imminent future where purported "artificial intelligence" which is just a plausible sentence generator but damn plausible alas will kill democracy and people. We are s…

The people who believe the vaxx bullshit don't read shit longer than a tweet anyway.

...the clairvoyant proclaimed confidently.

Re: Rogue superintelligence: Inside the mind of OpenAI's chief scientist

#46

Post by anonzzzies »

Cacti wrote: post #43
mattigames wrote: post #26

Earlier quoted context omitted.

The former president of the US is a glorified parrot, sometimes not even good at that, and still he was able to become president, so I wouldn't hold a defense for it to just that.

what does that have to do with anything?

That many (most?) humans, in power but more so outside are worthless parrots as well. And hallucinate whatever they don’t know or are unsure of. Even presidents. While using their native language terribly. It’s related to the LLMs taking over the world the ggp commented above: chatgpt is smarter than many humans I encounter daily, why couldn’t something just a tad better take over? If trump could, why not a stochastic parrot?

Re: Rogue superintelligence: Inside the mind of OpenAI's chief scientist

#48

Post by throwbadubadu »

> And he thinks some humans will one day choose to merge with machines. A lot of what Sutskever says is wild. But not nearly as wild as it would have sounded just one or two years ago.

Ok it is an intro.. but they say this as if he would be the first to say that, but that has been SciFi lore since computers were invented? And also as if this would not be happening today already at a certain limited scale.. so no doubts to this will happen at some point, if you count today's approaches not in.

Re: Rogue superintelligence: Inside the mind of OpenAI's chief scientist

#49

Post by peyton »

galoisscobi wrote: post #38

I, for one, am glad that Ilya has the reins on OpenAI and that Sam is out of the picture. It does seem that he weighs the ethics of what is being built more heavily than Sam. I’m also hoping that OpenAI cools down on the regulatory moat they were trying to build as a thinly veiled profit seeking strategy.

The OpenAI press release seemed to indicate explicitly that the 3 independent board members have the reins, not Ilya.

> The majority of the board is independent, and the independent directors do not hold equity in OpenAI.

Re: Rogue superintelligence: Inside the mind of OpenAI's chief scientist

#50

Post by feedforward »

vasco wrote: post #9

These people are too full of themselves. The physicists that invented The Bomb didn't have any special insight into the philosophical and societal implications. It takes a special kind of person to be able to invent something so big it can change the world, but it's about 0% chance that those people can control how the technology then gets used. I wish they'd focus more on the technical advances and less on trying to…

You are talking about the people not only doing the work and creating the wealth, but doing the innovations and creating new things. Then you say these are the people who are "too full of themselves". That they should give up on how the commodity they create is used - and by your definition it is a commodity. That they should focus on the needed innovations and that's it.

Well if the workers doing the work don't have a hand in making decisions, who does? We know the answer, from our current era of heirs, limited partners and such, the scions one can see on Rich Kids of Instagram, although they leave the work to their private wealth advisors.

It's the most parasitic idea possible. "Just do all the work and figure out the innovations slave, the aristocracy will take it from there".

Post reply on HN