Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

reuters.com

Post reply on HN
Print view View original

761–770 of 1001 posts

Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

#761

Post by qsort »

seanhunter wrote: post #735
nostrademons wrote: post #593

Earlier quoted context omitted.

What I wonder, as a computer scientist: If you want to solve grade school math problems, why not use an 'add' instruction? It's been around since the 50s, runs a billion times faster than an LLM, every assembly-language programmer knows how to use it, every high-level language has a one-token equivalent, and doesn't hallucinate answers (other than integer overflow). We also know how to solve complex reasoning chains…

There is a general result in machine learning known as "the bitter lesson"[1], which is that methods which come from specialist knowledge tend to be beaten by methods which rely on brute force computation in the long run because of Moore's law and the ability to scale things by distributed computing. So the reason people don't use the "add instruction"[2] for example is that over the last 70 years of attempting to bu…

As someone with a CS background myself, I don't think this is what GP was talking about.

Let's forget for a moment that stuff has to run on an actual machine. If you had to represent a quadratic equation, would you rather write:

(a) x^2 + 5x + 4 = 0

(b) the square of the variable plus five times the variable plus four equals zero

When you are trying to solve problems with a level of sophistication beyond the toy stuff you usually see in these threads, formal language is an aid rather than an impediment. The trajectory of every scientific field (math, physics, computer science, chemistry, even economics!) is away from natural language and towards formal language, even before computers, precisely for that reason.

We have lots of formal languages (general-purpose programming languages, logical languages like Prolog/Datalog/SQL, "regular" expressions, configuration languages, all kinds of DSLs...) because we have lots of problems, and we choose the representation of the problem that most suits our needs.

Unless you are assuming you have some kind of superintelligence that can automagically take care of everything you throw at it, natural language breaks down when your problem becomes wide enough or deep enough. In a way this is like people making Rube-Goldberg contraptions with Excel. 50% of my job is cleaning up that stuff.

Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

#763

Post by waveBidder »

gmt2027 wrote: post #749
wbhart wrote: post #491

I feel very comfortable saying, as a mathematician, that the ability to solve grade school maths problems would not be at all a predictor of ability to solve real mathematical problems at a research level. The reason LLMs fail at solving mathematical problems is because: 1) they are terrible at arithmetic, 2) they are terrible at algebra, but most importantly, 3) they are terrible at complex reasoning (more specifica…

We have an algorithm and computational hardware that will tune a universal function approximator to fit any dataset with emergent intelligence as it discovers abstractions, patterns, features and hierarchies. So far, we have not yet found hard limits that cannot be overcome by scaling the number of model parameters, increasing the size and quality of training data or, very infrequently, adopting a new architecture. T…

intelligence isn't a function unless you're talking about over every possible state of the universe.

Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

#764

Post by greendesk »

wbhart wrote: post #491

I feel very comfortable saying, as a mathematician, that the ability to solve grade school maths problems would not be at all a predictor of ability to solve real mathematical problems at a research level. The reason LLMs fail at solving mathematical problems is because: 1) they are terrible at arithmetic, 2) they are terrible at algebra, but most importantly, 3) they are terrible at complex reasoning (more specifica…

Thinking is about associations and object visualisation. Surely a non-human system can build those, right? Pointing out only to a single product exposed to the public does not prove limitations for a theoretical limit.

Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

#765

Post by chipsambos »

seanhunter wrote: post #716
oezi wrote: post #679

Earlier quoted context omitted.

What is the difference between simulating reasoning very well and "actual" reasoning?

Actual reasoning shows the understanding and use of a model of the key features of the underlying problem/domain. As a simple example that you can replicate using chatgpt, ask it to solve some simple maths problem. Very frequently you will get a solution that looks like reasoning but is not, and reveals that it does not have an actual model of the underlying maths but is in fact doing text prediction based on a histo…

I generally agree with what you're saying and the first half of your answer makes perfect sense but I think the second is unfair (i.e. "[is it] easier to balance a barrel on a plank or a plank on a barrel"). It's a trick question and "it" tried to answer in good faith.

If you were to ask the same question of a real person and they replied with the exact same answer you could not conclude that person was not capable of "actual reasoning". It's a bit of witch-hunt question set to give you the conclusion you want.

Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

#766

Post by dwroberts »

seydor wrote: post #730

i no longer follow the messianic complex of the people in openAI. They made great tech, indeed. Other people made great tech before them without instant religious level apocalypse proclamations. People in openAI are smart enough to know that post-AGI , their stock options are worthless anyway so they wouldn't stay walled in their secret garden if such a discovery had been made.

My guess is also in the opposite direction with this stuff: the Q breakthrough being mentioned here is phony in some way (beyond just being PR) and the ‘lack of candour’ referred to in the firing is failing to disclose how it actually worked after demoing it to the board (eg it uses humans in the loop or some other smoke and mirrors)

Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

#767

Post by ajuc »

silvaring wrote: post #698
oezi wrote: post #679

Earlier quoted context omitted.

What is the difference between simulating reasoning very well and "actual" reasoning?

Actual reasoning is made up of various biological feedback loops that happen in the body and brain, essentially your physical senses give you the ability to reason in the first place, without the eyes, ears etc there is no ability to learn basic reasoning, which is why kids who are blind or mute from birth have huge issues learning about object permanence, spatial awaraness etc. You cant expect human reasoning withou…

This ship can't swim because only living creatures swim. It's true but it only shows your definition sucks.

Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

#768

Post by vidarh »

nostrademons wrote: post #593
wbhart wrote: post #491

I feel very comfortable saying, as a mathematician, that the ability to solve grade school maths problems would not be at all a predictor of ability to solve real mathematical problems at a research level. The reason LLMs fail at solving mathematical problems is because: 1) they are terrible at arithmetic, 2) they are terrible at algebra, but most importantly, 3) they are terrible at complex reasoning (more specifica…

What I wonder, as a computer scientist: If you want to solve grade school math problems, why not use an 'add' instruction? It's been around since the 50s, runs a billion times faster than an LLM, every assembly-language programmer knows how to use it, every high-level language has a one-token equivalent, and doesn't hallucinate answers (other than integer overflow). We also know how to solve complex reasoning chains…

There's no value in an LLM doing arithmetic for the sake of doing arithmetic with the LLM. There's value in testing an LLMs ability to follow the rules for doing arithmetic that it already knows, because the ability to recognise that a problem matches a set of rules it already knows in part or whole and then applying those rules with precision is likely to generalise to overall far better problem solving abilities.

By all means, we should give LLMs lots and lots of specialised tools to let them take shortcuts, but that does not remove the reasons for understanding how to strengthen the reasoning abilities that would also make them good at maths.

EDIT: After having just coerced the current GPT4 to do arithmetic manually: It appears to have drastically improved in its ability to systematically following the required method, while ironically being far less willing to do so (it took multiple attempts before I got it to stop taking shortcuts that appeared to involve recognising this was a calculation it could use tooling to carry out, or ignoring my instructions to do it step by step and just doing it "in its head" the way a recalcitrant student might. It's been a while since I tested this, but this is definitely "new-ish".

Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

#769

Post by latexr »

Exoristos wrote: post #655
synaesthesisx wrote: post #151

Remember, about a month ago Sam posted a comment along the lines of "AI will be capable of superhuman persuasion well before it is superhuman at general intelligence, which may lead to very strange outcomes". The board was likely spooked by the recent breakthroughs (which were most likely achieved by combining transformers with another approach), and hit the panic button. Anything capable of "superhuman persuasion",…

Which party is "the wrong hands"?

The original commenter didn’t mention a party. Please don’t polarise the discussion into a flame war. Whatever system exists won’t be used by “a party” all at once, but by individuals. Any of those, with any political affiliation, can be “the wrong hands”.

I’ll offer a simple definition. The role of government is to serve the greater good of all people, thus the wrong hands are the ones which serve themselves or their own group above all.

Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

#770

Post by ben_w »

qsort wrote: post #761
seanhunter wrote: post #735

Earlier quoted context omitted.

There is a general result in machine learning known as "the bitter lesson"[1], which is that methods which come from specialist knowledge tend to be beaten by methods which rely on brute force computation in the long run because of Moore's law and the ability to scale things by distributed computing. So the reason people don't use the "add instruction"[2] for example is that over the last 70 years of attempting to bu…

As someone with a CS background myself, I don't think this is what GP was talking about. Let's forget for a moment that stuff has to run on an actual machine. If you had to represent a quadratic equation, would you rather write: (a) x^2 + 5x + 4 = 0 (b) the square of the variable plus five times the variable plus four equals zero When you are trying to solve problems with a level of sophistication beyond the toy stuf…

I assumed seanhunter was suggesting getting the LLM to convert x^2 + 5x + 4 = 0 to a short bit of source code to solve for x.

IIRC Wolfram Alpha has (or had, hard to keep up) a way to connect with ChatGPT.

Post reply on HN