OpenAI researchers warned board of AI breakthrough ahead of CEO ouster • Hacker News Board

Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

#901

Post by gryfft » Thu, Nov 23, 2023, 2:03 PM UTC

_heimdall wrote: post #857

gryfft wrote: post #362

Earlier quoted context omitted.

I'm as left leaning as HN commenters get, I think, and in terms of quacking like a duck, "surpassing humans in most economically valuable tasks" is 100% the meaningful Turing test in a capitalist society.

Take capitalism out of it, do we really want to boil down intelligence to a calculation of expected economic value? Why force the term intelligence into it at all if what were talking about is simply automation? We don't have to bastardize the term intelligence along the way, especially when we have spent centuries considering what human intelligence is and how it separates us from other species on the planet.

> Take capitalism out of it, do we really want to boil down intelligence to a calculation of expected economic value?

It's a lovely sentiment, but do you expect e.g. universities to start handing out degrees on the basis of human dignity rather than a series of tests whose ultimate purpose in our society is boiling down intelligence to a calculation of expected economic value?

We live in the world we live in and we have the measures we have. It's not about lofty ideals, it's about whether or not it can measurably do what a human does.

If I told you that my pillow is sentient and deserving of love and dignity, you have the choice of taking me at my word or finding a way to put my money where my mouth is. It's the same reason the world's best poker players aren't found by playing against each other with Monopoly money.

> Why force the term intelligence into it at all if what were talking about is simply automation?

In what world is modern AI "simply" anything?

> we have spent centuries considering what human intelligence is and how it separates us from other species on the planet.

Dolphins would like a word. There's more than a few philosophers who would argue that maybe our "intelligence" isn't so easily definable or special in the universe. There are zero successful capitalists who would pay my pillow or a dolphin to perform artificial intelligence research. That's what I mean when I call it the meaningful Turing test in a capitalist society. You can't just "take capitalism out of it." If I could just "take capitalism out" of anything meaningful I wouldn't be sitting here posting in this hell we're constructing. You may as well tell me to "take measurement out of it."

Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

#902

Post by anonymousDan » Thu, Nov 23, 2023, 2:11 PM UTC

jkeisling wrote: post #582

ssnistfajen wrote: post #142

So can people stop their cyberbullying campaign against the previous batch of board members yet? The conspiratorial hysteria against them, especially Helen Toner and Tasha McCauley, over the weekend was off the charts and straight up vile at times. Tech bros and tech bro wannabes continue to prove the necessity of shoving DEI into STEM disciplines even when it results in nonsensical wasted efforts, because they are i…

The "tech bros" were right: The board absolutely were conspirators wrecking the company based on dogmatic ideology. "Tech bros" had clear evidence of Sutskever, Toner and McCauley's deep and well-known ties to the Effective Altruist (EA) movement and doomerist views. Nobody can doubt Sutskever's technical credentials, whatever his strange beliefs, but Toner and McCauley had no such technical background or even busine…

'Dogmatic ideology' - I think you need to look in the mirror.

Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

#903

Post by wegfawefgawefg » Thu, Nov 23, 2023, 2:18 PM UTC

nabakin wrote: post #680

wegfawefgawefg wrote: post #343

If I had to guess, the name Q* is pronounced Q Star, and probably the Q refers to Q values or estimated rewards from reinforcement learning, and the star refers to a search and prune algorithm, like A* (A star). Possibly they combined deep reinforcement learning with self training and search and got a bot that could learn without needing to ingest the whole internet. Usually DRL agents are good at playing games, but…

https://youtu.be/PtAIh9KSnjo?t=3754

To give context on this video for anyone who doesn't understand. In this video PI* is referring to an idealized policy of actions that result in the maximum possible reward. (In Reinforcement Learning PI is just the actions you take in a situation). To use chess as an example, if you were to play the perfect move at every turn, that would be PI. Q is some function that tells you optimally, with perfect information the value of any move you could make. (Just like how stock-fish can tell you how many points a move in chess is worth.)

Now my personal comment: for games that are deterministic, there is no difference between a policy that takes the optimal move given only the current state of the board, and a policy that takes the optimal move given even more information, say a stack of future possible turns, etc.

However, in real life, you need to predict the future states, and sum across the best action taken at each future state as well. Unrealistic in the real world where the space of actions is infinite, and the universe to observe is not all simultaneously knowable. (hidden information)

Given the traditional educational background of the professionals in RL, maybe they were referring to the Q* from traditional rl. But I don't see why that would be novel, or notable, as it is a very old idea. Old Old math. From the 60s I think. So I sort of assumed its not. Could be relevant, or just a name collision.

Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

#904

Post by wegfawefgawefg » Thu, Nov 23, 2023, 2:25 PM UTC

GaryNumanVevo wrote: post #743

nabakin wrote: post #680

Earlier quoted context omitted.

https://youtu.be/PtAIh9KSnjo?t=3754

Man it's so sad to see how far Lex has fallen. From a graduate level guest lecturer at MIT to a glorified Joe Rogan

I could have given this lecture, and I think I could have made it much more entertaining, with fun examples.

Lex should stick to what he likes, though his interviews can be somewhat dull. On occasion I learn things from his guests I would have had no other chance of exposure to.

Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

#905

Post by wegfawefgawefg » Thu, Nov 23, 2023, 2:26 PM UTC

macrolime wrote: post #773

wegfawefgawefg wrote: post #343

If I had to guess, the name Q* is pronounced Q Star, and probably the Q refers to Q values or estimated rewards from reinforcement learning, and the star refers to a search and prune algorithm, like A* (A star). Possibly they combined deep reinforcement learning with self training and search and got a bot that could learn without needing to ingest the whole internet. Usually DRL agents are good at playing games, but…

I think more likely it's for finetuning a pre-trained model like GPT-4, kinda like RLHF, but in this case using reinforcement learning somewhat similar to AlphaZero. The model gets pre-trained and then fine-tuned to achieve mastery in tasks like mathematics and programming, using something like what you say and probably something like tree of thought and some self reflection to generate the data that it's using reinf…

I do not think that is correct as the RL in RLHF already stands for reinforcement learning. :^)

However, I do think you are right that self play, and something like reinforcement learning will be involved more in the future of ML. Traditional "data-first" ml has limits. Tesla conceded to RL for parking lots, where the action and state space was too unknowable for hand designed heuristics to work well. In Deep Reinforcement Learning making a model just copy data is called "behavior cloning", and in every paper I have seen it results in considerably worse peak performance than letting the agent learn from its own efforts.

Given that wisdom alone, we are under the performance ceiling with pure language models.

Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

#906

Post by muskmusk » Thu, Nov 23, 2023, 2:27 PM UTC

cduzz wrote: post #555

muskmusk wrote: post #503

Earlier quoted context omitted.

If you ask Ilya Sutskever he will say your kids head is full of neurons, so is LLMs. LLMs comsume training data and can then be asked questions. How different is that to your son watching YouTube and then answering questions? It's not 1:1 the same,yet, but it's in the neighborhood.

Well, my son is a meat robot who's constantly ingesting information from a variety of sources including but not limited to youtube. His firmware includes a sophisticated realtime operating system that models reality in a way that allows interaction with the world symbolically. I don't think his solving the |i+1| question was founded in linguistic similarity but instead in a physical model / visualization similarity.…

Heh I guess it's s matter of perspective. Your son's head is not made of silicon so in that sense it is a large neighborhood. But if you put them behind a screen and only see the output then the neighborhood looks smaller. Maybe it looks even smaller a couple of years in the future. It certainly looks smaller than it did a couple of years in the past.

Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

#907

Post by yallneedtoget » Thu, Nov 23, 2023, 2:28 PM UTC

wbhart wrote: post #491

I feel very comfortable saying, as a mathematician, that the ability to solve grade school maths problems would not be at all a predictor of ability to solve real mathematical problems at a research level. The reason LLMs fail at solving mathematical problems is because: 1) they are terrible at arithmetic, 2) they are terrible at algebra, but most importantly, 3) they are terrible at complex reasoning (more specifica…

[dead]

Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

#908

Post by oblio » Thu, Nov 23, 2023, 2:29 PM UTC

BenoitP wrote: post #869

passwordoops wrote: post #860

Earlier quoted context omitted.

These guys smell so much like Tesla it's not even funny. Very impressive core tech, and genuinely advancing knowledge. But the hype train is just so powerful that the insane (to put it mildly) claims are picked up without any sense of critical thinking by seemingly intelligent people. They're both essentially cults at this point

Agreed, but IMHO it is sort of justified for Tesla. The size of its hype matches the size of the ICEs entrenchment in its moat. These have outsize influence on our economy, but climate change (and oil depletion) is quite inevitable. It takes irrational market cap to unseat that part of the economy being prisoner of its rent. And some allocators of capital have understood that.

Check the left hand side menu: https://www.arenaev.com/

The list of ICE car manufacturers making EVs is longer than my arm. All the European ones have staked their future on EVs. I think VW (the irony) was lobbying for a faster phaseout of ICEs in Europe, because they're well positioned to take over the EV market if ICEs are banned faster than 2035 :-)

Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

#909

Post by dallama » Thu, Nov 23, 2023, 2:35 PM UTC

jiggawatts wrote: post #727

wbhart wrote: post #491

I feel very comfortable saying, as a mathematician, that the ability to solve grade school maths problems would not be at all a predictor of ability to solve real mathematical problems at a research level. The reason LLMs fail at solving mathematical problems is because: 1) they are terrible at arithmetic, 2) they are terrible at algebra, but most importantly, 3) they are terrible at complex reasoning (more specifica…

Everything you said about LLMs being "terrible at X" is true of the current generation of LLM architectures. From the sound of it, this Q* model has a fundamentally different architecture, which will almost certainly make some of those issues not terrible any more. Most likely, the Q* design is the very similar to the one suggested recently by one of the Google AI teams: doing a tree search instead of greedy next tok…

I immediately thought of A* path finding, I'm pretty sure Q* is the LLM "equivalent". Much like you describe.

Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

#910

Post by throwuwu » Thu, Nov 23, 2023, 2:35 PM UTC

nostrademons wrote: post #593

wbhart wrote: post #491

I feel very comfortable saying, as a mathematician, that the ability to solve grade school maths problems would not be at all a predictor of ability to solve real mathematical problems at a research level. The reason LLMs fail at solving mathematical problems is because: 1) they are terrible at arithmetic, 2) they are terrible at algebra, but most importantly, 3) they are terrible at complex reasoning (more specifica…

What I wonder, as a computer scientist: If you want to solve grade school math problems, why not use an 'add' instruction? It's been around since the 50s, runs a billion times faster than an LLM, every assembly-language programmer knows how to use it, every high-level language has a one-token equivalent, and doesn't hallucinate answers (other than integer overflow). We also know how to solve complex reasoning chains…

What you’re proposing is equivalent to training a monkey (or a child for that matter) to punch buttons that correspond to the symbols it sees without actually teaching it what any of the symbols mean.