Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

reuters.com

Post reply on HN
Print view View original

841–850 of 1001 posts

Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

#841

Post by setuid9002 »

nostrademons wrote: post #593
wbhart wrote: post #491

I feel very comfortable saying, as a mathematician, that the ability to solve grade school maths problems would not be at all a predictor of ability to solve real mathematical problems at a research level. The reason LLMs fail at solving mathematical problems is because: 1) they are terrible at arithmetic, 2) they are terrible at algebra, but most importantly, 3) they are terrible at complex reasoning (more specifica…

What I wonder, as a computer scientist: If you want to solve grade school math problems, why not use an 'add' instruction? It's been around since the 50s, runs a billion times faster than an LLM, every assembly-language programmer knows how to use it, every high-level language has a one-token equivalent, and doesn't hallucinate answers (other than integer overflow). We also know how to solve complex reasoning chains…

I think the answer is Money, Money, Money. Sure it is 1000000000x more expensive in compute power, and error prown on top as well, to let a LLM solve an easy Problem. But the Monopolies generate a lot of hype around it to get more money from investors. Same as the self driving car hype was. Or the real time raytracing insanity in computer graphics. If one hype dies they artificially generate a new one. For me, I just watch all the ships sink to the ground. It is gold level comedy. Btw AGI is coming, haha, sure, we developers will be replaced by an script which can not bring B, A, C in a logical sequence. And this already needs massive town size data centers to train.

Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

#842

Post by Enogoloyo »

wbhart wrote: post #491

I feel very comfortable saying, as a mathematician, that the ability to solve grade school maths problems would not be at all a predictor of ability to solve real mathematical problems at a research level. The reason LLMs fail at solving mathematical problems is because: 1) they are terrible at arithmetic, 2) they are terrible at algebra, but most importantly, 3) they are terrible at complex reasoning (more specifica…

Just wait a little bit.

You are not better than a huge GPU cluster with Monte Carlo search and computer verification for much longer.

It will be more your job to find the interesting finds than doing the work of finding things in the first olace

Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

#844

Post by Robin_Message »

marvin wrote: post #671
Atheros wrote: post #603

Earlier quoted context omitted.

And what if there are no alignment methods.

Yudkowsky’s doomsday cult almost blew OpenAI to pieces and sent everyone who knows the details in the wind like dandelion seeds. What’s next? A datacenter bombing or killing key researchers? We should be happy that this particular attempt failed, because this cult is only capable of strategic actions that make things far more dangerous. This will be solved like all other engineering and science: with experiments and…

> It’s bad for the bottom line to be dead.

I have no overall position, but climate change and nuclear weapons seem two quite strong counterexamples to this being a sufficient condition for safety.

Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

#845

Post by LightMachine »

reasonableklout wrote: post #164
V__ wrote: post #3

> warning of a powerful artificial intelligence discovery that they said could threaten humanity, > Given vast computing resources, the new model was able to solve certain mathematical problems, [..] Though only performing math on the level of grade-school students, acing such tests made researchers very optimistic about Q*’s future success, the source said. I somehow expected a bit more.

I don't really understand this. Aren't LLMs already performing at near-expert level on "certain mathematical problem" benchmarks? For example, over a year ago MINERVA from Google [1] got >50% on the MATH dataset, a set of competition math problems. These are not easy problems. From the MATH dataset paper: > We also evaluated humans on MATH, and found that a computer science PhD student who does not especially like ma…

No, they are completely awful at math. The way you can see that is that, whenever you ask it about known concepts, even hard ones, it will answer, making it look intelligent. But then if you create a small new theory or logical problem that is completely not in the internet, and ask SIMPLE questions about it - i.e., questions a 10 years old would be able to answer given the context - it will fail disgracefully.

Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

#846

Post by afpx »

wbhart wrote: post #491

I feel very comfortable saying, as a mathematician, that the ability to solve grade school maths problems would not be at all a predictor of ability to solve real mathematical problems at a research level. The reason LLMs fail at solving mathematical problems is because: 1) they are terrible at arithmetic, 2) they are terrible at algebra, but most importantly, 3) they are terrible at complex reasoning (more specifica…

What amazes me is how close it gets to the right answer, though. Pick a random 10-digit number, then ask the next 20 numbers in sequence.

I feel like the magic in these LLMs is in how they work well in stacks, trees or in seqence. They become elements of other data structures. Consider a network of these, combined with other specialized systems and an ability to take and give orders. With reinforcement learning, it could begin building better versions of itself.

Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

#847

Post by walthamstow »

Sol- wrote: post #832
JacobJeppesen wrote: post #772

Seems like they have made progress in combining reinforcement learning and LLMs. Andrej Karpathy mentions it in his new talk (~38 minutes in) [1], and Ilya Sutskever talks about it in a lecture at MIT (~29 minutes in) [2]. It would be a huge breakthrough to find a proper reward function to train LLMs in a reinforcement learning setup, and to train a model to solve math problems in a similar fashion to how AlphaGo use…

Thanks for the links, very interesting. Wonder how a "self-play" equivalent would look like for LLMs, since they have no easy criterion to evaluate how well they are doing like in Go (as mentioned in the videos).

ChatGPT does have some feedback that can be used to evaluate, in the form of thumbs up/down buttons, which probably nobody uses, and positive/negative responses to its messages. People often say "thanks" or "perfect!" in responses, including very smart people who frequent here.

Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

#848

Post by naikrovek »

wbhart wrote: post #491

I feel very comfortable saying, as a mathematician, that the ability to solve grade school maths problems would not be at all a predictor of ability to solve real mathematical problems at a research level. The reason LLMs fail at solving mathematical problems is because: 1) they are terrible at arithmetic, 2) they are terrible at algebra, but most importantly, 3) they are terrible at complex reasoning (more specifica…

it's always nice to see HN commenters with so much confidence in themselves that they feel they know a situation better than the people who are actually in the situation being discussed.

Do you really believe that they don't have skilled people on staff?

Do you really believe that your knowledge of what OpenAI is doing is a superset of the knowledge of the people who work at OpenAI?

give me 0.1% of your confidence and I would be able to change the world.

Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

#849

Post by Levitz »

oezi wrote: post #679
poulpy123 wrote: post #667

Earlier quoted context omitted.

I don't know for Q* of course, but all the tests I made with GPT4, and all what I've read and seen about it, show that it is unable to reason. It was trained with an unfathomable amount of data, so it can simulate reasoning very well, but it is unable to reason

What is the difference between simulating reasoning very well and "actual" reasoning?

Being able to extrapolate with newly found data.

You can get a LLM to simulate it "discovering" the pythagorean theorem, but can it actually, with the knowledge that was available at the time, discover the pythagorean theorem by itself?

Any parent will tell you, it's easy to simulate discovery and reasoning, it's a trick played for kids all the time. The actual, real stuff, that's way harder.

Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

#850

Post by manx »

Sol- wrote: post #832
JacobJeppesen wrote: post #772

Seems like they have made progress in combining reinforcement learning and LLMs. Andrej Karpathy mentions it in his new talk (~38 minutes in) [1], and Ilya Sutskever talks about it in a lecture at MIT (~29 minutes in) [2]. It would be a huge breakthrough to find a proper reward function to train LLMs in a reinforcement learning setup, and to train a model to solve math problems in a similar fashion to how AlphaGo use…

Thanks for the links, very interesting. Wonder how a "self-play" equivalent would look like for LLMs, since they have no easy criterion to evaluate how well they are doing like in Go (as mentioned in the videos).

One could generate arbitrarily many math problems, where the solution is known.
Post reply on HN