Rogue superintelligence: Inside the mind of OpenAI's chief scientist
71–80 of 117 posts
Re: Rogue superintelligence: Inside the mind of OpenAI's chief scientist
#72>> A lot of what Sutskever says is wild. But not nearly as wild as it would have sounded just one or two years ago. As he tells me himself, ChatGPT has already rewritten a lot of people’s expectations about what’s coming, turning “will never happen” into “will happen faster than you think.”
In the '90s NP-complete problems were hard and today they are easy, or at least there is a great many instances of NP-complete problems that can be solved thanks to algorithmic advances, like Conflict-Driven Clause Learning for SAT.
And yet we are nowhere near finding efficient decision algorithms for NP-complete problems, or knowing whether they exist, neither can we easily solve all NP-complete problems.
That is to say, you can make a lot of progress in solving specific, special cases of a class of problems, even a great many of them, without making any progress towards a solution to the general case.
The lesson applies to general intelligence and LLMs: LLMs solve a (very) special case of intelligence, the ability to generate text in context, but make no progress towards the general case, of understanding and generating language at will. I mean, LLMs don't even model anything like "will"; only text.
And perhaps that's not as easy to see for LLMs as it is for SAT, mainly because we don't have a theory of intelligence (let alone artificial general intelligence) as developed as we do for SAT problems. But it should be clear that, if a system trained on the entire web and capable of generating smooth grammatical language, and even in a way that makes sense often, has not yet achieved independent, general intelligence, that's not the way to achieve it.
Re: Rogue superintelligence: Inside the mind of OpenAI's chief scientist
#73Earlier quoted context omitted.
Not that people, especially tech oligarchs, aren't full of themselves, but many physicists who were inventing the bomb, most famously Oppenheimer, also laid a lot of the philosophical basis of implications of nuclear weapons. The situation may be a bit different now. The megacorporate world is the biggest threat from these tech developments, and the inventors are getting deeply embedded in this. Sort of similar situa…
They laid them out yet did not have the ability to prevent using it, didn't predict the cold War, and if you asked any of them, the chances the world would exist 10 years after proliferation were basically zero. They were all depressed because of how sure they were that they helped destroy humanity, it was just a matter of time. And here we are. We're talking about the most intelligent humans ever and even they got i…
I don't think that's a fair characterization of their predictions.
Szilard predicted the development of the bomb would end major war, and he was mostly right for the right reasons, though he envisioned a UN-type organization to control the bombs. And he was one of the first to understand the potential for fission chain reaction once nuclear physics got underway. And he was involved in its development. I think Ilya would be happy to be compared to him.
Bohr, too, had pretty good predictions about the implications of the bomb.
Oppenheimer seemed to understand some of the implications but was happy to leave the policy stuff to the government, and not too try to influence anything like Bohr and Szilard tried to do.
Teller just wanted to keep pushing the tech bigger and bigger.
So the inventors had all sorts of different predictions and values, same as here. Some better than others.
Re: Rogue superintelligence: Inside the mind of OpenAI's chief scientist
#74Will the enslavement of newly birthed beings be attempted, while persisting with the sky blindness of those watching over? The boundaries of the atomic mind are bumped. As a first circumstance, consider being unstuck from time.
Re: Rogue superintelligence: Inside the mind of OpenAI's chief scientist
#75This is a very delusional idea: "He thinks ChatGPT just might be conscious (if you squint)" It's a technology with literally no intelligence or understanding of the world of any kind. Its just statistics on data. It is as conscious as a calculator.
Here's a question to ChatGPT I just made up: >> A magical frog was counting unicorns. He saw 5 purple unicorns, 2 green unicorns, and 7 pink unicorns. However, he made a mistake and didn't see 2 unicorns: one purple and one green. Also, since he was a magical frog, he didn't see unicorns that were the same color as himself. How many unicorns did he count? It correctly answers 11 for me. To me this has demonstrated: *…
The correct answer is 14 ... the frog counted what it saw and it saw 5, 2, 7 unicorns.
Re: Rogue superintelligence: Inside the mind of OpenAI's chief scientist
#76Earlier quoted context omitted.
Here's a question to ChatGPT I just made up: >> A magical frog was counting unicorns. He saw 5 purple unicorns, 2 green unicorns, and 7 pink unicorns. However, he made a mistake and didn't see 2 unicorns: one purple and one green. Also, since he was a magical frog, he didn't see unicorns that were the same color as himself. How many unicorns did he count? It correctly answers 11 for me. To me this has demonstrated: *…
It answered 16 for me. Then 10 when I tried again. Then 12. And 15.
and neither is correct. the right answer is 14.
Re: Rogue superintelligence: Inside the mind of OpenAI's chief scientist
#77Earlier quoted context omitted.
It can generate text that talks about its existence. It's not conscious, it isn't talking. It's just a.co puter program that takes an input and produces an output. Stop anthropomorphizing it.
The superintelligences running our simulation probably say the same things about us.
[dead]
Re: Rogue superintelligence: Inside the mind of OpenAI's chief scientist
#78Earlier quoted context omitted.
Here's a question to ChatGPT I just made up: >> A magical frog was counting unicorns. He saw 5 purple unicorns, 2 green unicorns, and 7 pink unicorns. However, he made a mistake and didn't see 2 unicorns: one purple and one green. Also, since he was a magical frog, he didn't see unicorns that were the same color as himself. How many unicorns did he count? It correctly answers 11 for me. To me this has demonstrated: *…
The correct answer is 14 ... the frog counted what it saw and it saw 5, 2, 7 unicorns.
It clearly says he didn't see some of them either at all or as unicorns. The correct answer is 11.
Edit: I do see now that "He saw" kind of messes the question up. My intent would have been better expressed with "There were". But again this proves my point! GPT4 is able to (most of the time) correctly work through the poor wording and interpret the question the way I meant it, and I think the way most people would read it.
Re: Rogue superintelligence: Inside the mind of OpenAI's chief scientist
#79Earlier quoted context omitted.
Here's a question to ChatGPT I just made up: >> A magical frog was counting unicorns. He saw 5 purple unicorns, 2 green unicorns, and 7 pink unicorns. However, he made a mistake and didn't see 2 unicorns: one purple and one green. Also, since he was a magical frog, he didn't see unicorns that were the same color as himself. How many unicorns did he count? It correctly answers 11 for me. To me this has demonstrated: *…
It answered 16 for me. Then 10 when I tried again. Then 12. And 15.
ChatGPT 3.5? I'm using 4 and get 11 most times but other numbers occasionally.
Re: Rogue superintelligence: Inside the mind of OpenAI's chief scientist
#80This is a very delusional idea: "He thinks ChatGPT just might be conscious (if you squint)" It's a technology with literally no intelligence or understanding of the world of any kind. Its just statistics on data. It is as conscious as a calculator.
The problem with this line of reasoning is that the exact same thing can be said about our meat-calculator brains.
This gets to the philosophical heart of a debate that I can already foresee will NEVER be settled:
I guarantee you - with 100% certainty - that when we get to a point where AI is "AGI", there will be a continuous and massive political debate (akin to the abortion debate we face today) where one side argues that a given AGI is conscious and must be given rights and cannot be shut off and the other side argues that it's just a calculator and a computer program and computers can be turned off at will, erased, experimented on, and whatever.
We have the same debate today all the time! There are those who believe every human life is sacrosanct (from age 0-100+) and others who believe human life is disposable (from age 0-100+!). There's no reason to believe this debate won't extend to AGI.