Rogue superintelligence: Inside the mind of OpenAI's chief scientist

Because they were not charge, if anything the more aggressive voices among them had bigger roles in the cold war. You can never rule some impact, but there was an awful lot of effort going into handling cold war dynamics where those scientists had no role in.

This line of nonsense has become the new "Tell me you aren't current on the past 12 months of LLM research without telling me."

Harvard/MIT's Othello-GPT paper showing the development of what turned out to be linear representations of world models from training data that didn't explicitly contain that modeling is over a year old now.

That in turn inspired research showing linear representations in geographical mapping and in more traditional text models around truthiness vs falsehoods.

So we already have an increasing research trend that is showing over and over linear representations of more abstract modeling than "just statistics."

So you are wrong that LLMs with sufficient network complexity don't develop an understanding of the world (in parts).

And I'd encourage looking more into the difference between understanding the difference between training for next token prediction and the overall capabilities of the network with the smallest loss at that training task, particularly as network complexity increases.

David chalmers doesn't think llm's are conscious yet but also doesn't think there's any real barriers for silicon to get to human consciousness[0]

[0]https://youtu.be/j6cCXg-rjRo

He gives a few of the biggest reasons they're not conscious, and gives his thoughts on them as long term barriers.

1. biology necessary for life? He doesn't buy it

2. sensors/body - no embodiment akin to philosophers brain in a vat at best. "no agentive consciousness" he also doesn't buy this one, a it's solvable, b the brain in a vat might still have some limited form of consciousness

3.world models : stochastic parrot argument - he doesn't buy this either as being a valid "llm's will never be conscious" reason. he even thinks current llm's did have some world model if not a "full" one"

4. feed forward systems - stateless - lack long term memory Recurrent processing necessary? "Not all consciousness involves memory"

5. Unified agency - they're chameleons, actors, - are stable goals/unity necessary for consciousness? a "fixed" identity?

from a quick scroll thru of the vid

OpenAI is a religious institution that self-selects for fundamentalism. This is why so many very good researchers passed on working there, they simply didn’t believe in The Mission Where Ilya Saves Humanity.

This is why you need to take classes other than computers and math, kids.

Here's a question to ChatGPT I just made up:

>> A magical frog was counting unicorns. He saw 5 purple unicorns, 2 green unicorns, and 7 pink unicorns. However, he made a mistake and didn't see 2 unicorns: one purple and one green. Also, since he was a magical frog, he didn't see unicorns that were the same color as himself. How many unicorns did he count?

It correctly answers 11 for me.

To me this has demonstrated:

* "Understanding": It understood that "didn't see" implies he didn't count.

* "Knowledge": It knew enough about the world to know that frogs are often green.

* "Reasoning": It was able to correctly reason about how many should be subtracted from the final result.

* "Math: It successfully did some basic additions and subtractions arriving at the correct answer.

Crucially, I made this up right here on the spot, and used a dice for some of the numbers. This question does not exist anywhere in the training corpus!

I think this demonstrates an impressive level of intelligence, for what up until about a year ago I thought a computer would ever be capable of in my lifetime. Now in absolute terms of course current gen ChatGPT is clearly far less good at reasoning and understanding than most people (well, specifically it seems to me that it's knowledge and reasoning are super-humanly broad, but child-level deep).

Can future improvements to this architecture improve the depth up to "AGI", whatever that means? I have no idea. It doesn't automatically seem impossible, but maybe what we see now is already near the limit? I guess only time will tell.