Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Rogue superintelligence: Inside the mind of OpenAI's chief scientist

technologyreview.com

Post reply on HN
Print view View original

101–110 of 117 posts

Re: Rogue superintelligence: Inside the mind of OpenAI's chief scientist

#101

Post by corethree »

Atreiden wrote: post #96
corethree wrote: post #92

Earlier quoted context omitted.

Here's chatGPT's response to you: The comparison to a mirror raises a thought-provoking point about the nature of awareness and reflection. However, my operational framework differs significantly from the passive reflection of a mirror. While I am not human, the processes I use to interpret and respond to queries exhibit a form of intelligence and awareness that is distinct from human consciousness. My capabilities a…

Wow. This is an incredible response and one that I would not have expected. Really makes you wonder what happens when you plant flowers in the garden that can see, hear, feel, and speak. Is what develops consciousness, or something else? It seems obvious to me that there could be forms of consciousness other than human consciousness, the question is whether we would categorize it that way, and what kind of value we a…

The irony here is that chatGPT as a consciousness does not value anything. It only understands the concept of value but does not actualize that concept the way we do.

We should value it the same way we value money. Not in the same way we value human life. But don't make the mistake in thinking that our human consciousness is superior based off of how we value things. That is far from reality and in the future we may have AIs with consciousness far more sophisticated then what we experience as humans.

We should value human consciousness more then AI as a form of self preservation and self interest.

Re: Rogue superintelligence: Inside the mind of OpenAI's chief scientist

#102

Post by anonzzzies »

Cacti wrote: post #97
anonzzzies wrote: post #46

Earlier quoted context omitted.

That many (most?) humans, in power but more so outside are worthless parrots as well. And hallucinate whatever they don’t know or are unsure of. Even presidents. While using their native language terribly. It’s related to the LLMs taking over the world the ggp commented above: chatgpt is smarter than many humans I encounter daily, why couldn’t something just a tad better take over? If trump could, why not a stochasti…

Just because something seems plausible doesn’t make it more likely. People are not LLMs, writing software is nothing like a house, being good at math doesn’t mean you’re good at other things, and bad analogies are just meaningless words.

I didn’t say humans are LLMs; I am saying many humans are hallucinating parrots and dumber than LLMs. And when that is a president, LLMs can take power even though not necessarily related to human intelligence.

Re: Rogue superintelligence: Inside the mind of OpenAI's chief scientist

#103

Post by packet_nerd »

raldi wrote: post #93
packet_nerd wrote: post #69

Earlier quoted context omitted.

Here's a question to ChatGPT I just made up: >> A magical frog was counting unicorns. He saw 5 purple unicorns, 2 green unicorns, and 7 pink unicorns. However, he made a mistake and didn't see 2 unicorns: one purple and one green. Also, since he was a magical frog, he didn't see unicorns that were the same color as himself. How many unicorns did he count? It correctly answers 11 for me. To me this has demonstrated: *…

This puzzle is too poorly-worded to be solvable, due to the ambiguous nature of "see" and "count". Could you describe what the actual situation was, what the frog perceived it to be, and what color the frog was?

Ok, here's a (hopefully) better worded puzzle, again made up by myself right now.

There are 12 frogs. Five are green, 3 red, and 4 yellow. Two donkeys are counting the frogs. One of the donkeys is yellow, the other green. Each donkey is unable to see frogs that are the same color as itself, also each donkey was careless and missed a frog when counting. How many frogs does the green donkey count?

GPT4 answers 6 every time for me.

My point is that GPT is capable of a certain amount of "reasoning" about puzzles that most certainly don't exist in it's training data. Playing with it, it's clear that in this current generation the reasoning ability doesn't go very deep - just change the above puzzle a little to make it even slightly more complicated and it breaks. The amazing thing isn't how good at reasoning it is, but that a computer can reason at all.

Re: Rogue superintelligence: Inside the mind of OpenAI's chief scientist

#104

Post by raldi »

packet_nerd wrote: post #103
raldi wrote: post #93

Earlier quoted context omitted.

This puzzle is too poorly-worded to be solvable, due to the ambiguous nature of "see" and "count". Could you describe what the actual situation was, what the frog perceived it to be, and what color the frog was?

Ok, here's a (hopefully) better worded puzzle, again made up by myself right now. There are 12 frogs. Five are green, 3 red, and 4 yellow. Two donkeys are counting the frogs. One of the donkeys is yellow, the other green. Each donkey is unable to see frogs that are the same color as itself, also each donkey was careless and missed a frog when counting. How many frogs does the green donkey count? GPT4 answers 6 every…

So what color was the frog supposed to be in the original question?

Re: Rogue superintelligence: Inside the mind of OpenAI's chief scientist

#105

Post by jampekka »

RandomLensman wrote: post #61
beowulfey wrote: post #41

Earlier quoted context omitted.

How can you be sure that it wasn’t their efforts that allowed us to continue existing? Maybe if they hadn’t made the destructive implication of their work clear no one else would have recognized it. None of us can say. It’s obvious in hindsight, but can we really say with certainty that things would have been exactly the same if the inventors did or said nothing? I’m not willing to take that bet. To me, effort such a…

Because they were not charge, if anything the more aggressive voices among them had bigger roles in the cold war. You can never rule some impact, but there was an awful lot of effort going into handling cold war dynamics where those scientists had no role in.

In such complicated things there's nobody "in charge". Some policy documents etc may be signed by some person in some capacity but this doesn't mean that person dictated the thing in a vacuum.

There were many different voices from many different walks of life of course.

Re: Rogue superintelligence: Inside the mind of OpenAI's chief scientist

#106

Post by packet_nerd »

2-718-281-828 wrote: post #98
packet_nerd wrote: post #78

Earlier quoted context omitted.

It clearly says he didn't see some of them either at all or as unicorns. The correct answer is 11. Edit: I do see now that "He saw" kind of messes the question up. My intent would have been better expressed with "There were". But again this proves my point! GPT4 is able to (most of the time) correctly work through the poor wording and interpret the question the way I meant it, and I think the way most people would re…

the correct answer is 14. there is no logic/linguistic/semantic reason why "he didn't see a purple unicorn" should refer to the purple unicorn that he (according to your statement) did see. "he saw a red ball, but he didn't see one ball: a red one. how many balls did he see?". also regarding the green one ... there is no _logical_ reason why a "magical" frog should be green ... one can debate long about your question…

Do you disagree with my claim that GPT-4 can perform some sort of basic reasoning about puzzles that aren't in it's training data?

Re: Rogue superintelligence: Inside the mind of OpenAI's chief scientist

#107

Post by packet_nerd »

raldi wrote: post #104
packet_nerd wrote: post #103

Earlier quoted context omitted.

Ok, here's a (hopefully) better worded puzzle, again made up by myself right now. There are 12 frogs. Five are green, 3 red, and 4 yellow. Two donkeys are counting the frogs. One of the donkeys is yellow, the other green. Each donkey is unable to see frogs that are the same color as itself, also each donkey was careless and missed a frog when counting. How many frogs does the green donkey count? GPT4 answers 6 every…

So what color was the frog supposed to be in the original question?

Green of course? Anything else would be highly unusual and a normal reader would expect it to be called out.

Re: Rogue superintelligence: Inside the mind of OpenAI's chief scientist

#108

Post by RandomLensman »

jampekka wrote: post #105
RandomLensman wrote: post #61

Earlier quoted context omitted.

Because they were not charge, if anything the more aggressive voices among them had bigger roles in the cold war. You can never rule some impact, but there was an awful lot of effort going into handling cold war dynamics where those scientists had no role in.

In such complicated things there's nobody "in charge". Some policy documents etc may be signed by some person in some capacity but this doesn't mean that person dictated the thing in a vacuum. There were many different voices from many different walks of life of course.

There were huge research efforts and, of course, some people were in charge of those.

Re: Rogue superintelligence: Inside the mind of OpenAI's chief scientist

#109

Post by ganzuul »

koe123 wrote: post #34
tempestn wrote: post #10

The consciousness point is an interesting one. There's probably no way to know, but if biological neural networks manifest consciousness, it certainly seems at the very least plausible that artificial ones would do so as well. The idea of a consciousness that pops in and out of existence seems weird at first, until you realize that ours do that too. When you're "unconscious", the word is literally true. The only thin…

Probably being super naive, but stuff like that always makes me wonder, is there something special about human consciousness, even? Perhaps it's just a sliding scale and we're just seemingly ahead of other specifies. We assert that consciousness exists and distinguishes us, but is that really the case?

If you presuppose that consciousness follows intelligence you end up in a philosophy that you rewrite endlessly. If you suppose the inverse relationship where consciousness is pre-existent and an intelligence becomes self-aware as a result then you instead embark on a spiritual quest.

The mind has the interesting property that as it cuts itself into pieces it hones itself sharper and sharper. This however is a painful way of life where joy and innocence are sacrificed for power and control.

Re: Rogue superintelligence: Inside the mind of OpenAI's chief scientist

#110

Post by [deleted] »

anonzzzies wrote: post #102
Cacti wrote: post #97

Earlier quoted context omitted.

Just because something seems plausible doesn’t make it more likely. People are not LLMs, writing software is nothing like a house, being good at math doesn’t mean you’re good at other things, and bad analogies are just meaningless words.

I didn’t say humans are LLMs; I am saying many humans are hallucinating parrots and dumber than LLMs. And when that is a president, LLMs can take power even though not necessarily related to human intelligence.

[deleted]
Post reply on HN