Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

reuters.com

Post reply on HN
Print view View original

561–570 of 1001 posts

Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

#561

Post by insomagent »

wbhart wrote: post #491

I feel very comfortable saying, as a mathematician, that the ability to solve grade school maths problems would not be at all a predictor of ability to solve real mathematical problems at a research level. The reason LLMs fail at solving mathematical problems is because: 1) they are terrible at arithmetic, 2) they are terrible at algebra, but most importantly, 3) they are terrible at complex reasoning (more specifica…

Let's say a model runs through a few iterations and finds a small, meaningful piece of information via "self-play" (iterating with itself without further prompting from a human.)

If the model then distills that information down to a new feature, and re-examines the original prompt with the new feature embedded in an extra input tensor, then repeats this process ad-infinitum, will the language model's "prime directive" and reasoning ability be sufficient to arrive at new, verifiable and provable conjectures, outside the realm of the dataset it was trained on?

If GPT-4,5,...,n can progress in this direction, then we should all see the writing on the wall. Also, the day will come where we don't need to manually prepare an updated dataset and "kick off a new training". Self-supervised LLMs are going to be so shocking.

Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

#563

Post by maxglute »

gexla wrote: post #336
maxglute wrote: post #210

Earlier quoted context omitted.

I haven't followed the situation as closely as others, but it does seem like board structured to not have 7 figure cheques that could influence / undermine safety mission seemingly willing to let openAI burn out of dogma. Employees want their 7 figure cheques, their interests aligned with deeper pockets and larger powers with 10+ figures on the line. Reporting so far have felt biased accordingly. If this was about mo…

Seems like the board's mission went off the rails long ago and it acted late. Some snippets from the OpenAI website... "Investing in OpenAI Global, LLC is a high-risk investment" "Investors could lose their capital contribution and not see any return" "It would be wise to view any investment in OpenAI Global, LLC in the spirit of a donation, with the understanding that it may be difficult to know what role money will…

>should have accepted slow progress

Sure, should have. I think indicators in the last year have pointed to the domain developing much faster than anticipated, with Ilya seemingly incredulous at how well models he spend his career developing, suddenly started working and scaling incredibly well. If they thought billions of "donations" would sustain development in commercial capabilities X within constraints of the mission, but got X^10 way outside constraints, and their explicit goal was to to make sure X^10 doesn't arrive without Y^10 in consideration for safety, it's reasonable for hard liners to reevaluate, and if forces behind the billions get in the way, to burn it all down.

Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

#564

Post by somenameforme »

synaesthesisx wrote: post #151

Remember, about a month ago Sam posted a comment along the lines of "AI will be capable of superhuman persuasion well before it is superhuman at general intelligence, which may lead to very strange outcomes". The board was likely spooked by the recent breakthroughs (which were most likely achieved by combining transformers with another approach), and hit the panic button. Anything capable of "superhuman persuasion",…

It seems much more likely that this was just referring to the ongoing situation with LLMs being able to create exceptionally compelling responses to questions that are completely and entirely hallucinated. It's already gotten to the point that I simply no longer use LLMs to learn about topics I am not already extremely familiar with, simply because hallucinations end up being such a huge time waster. Persuasion without accuracy is probably more dangerous to their business model than the world, because people learn extremely quickly not to use the models for anything you care about being right on.

Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

#565

Post by wbhart »

neilk wrote: post #532
muskmusk wrote: post #509

Earlier quoted context omitted.

Friend, the creator of this new progress is a machine learning PhD with a decade of experience in pushing machine learning forward. He knows a lot of math too. Maybe there is a chance that he too can tell the difference between a meaningless advance and an important one?

I am neither a mathematician or LLM creator but I do know how to evaluate interesting tech claims. The absolute best case scenario for a new technology is that it when it seems like a toy for nerds, and doesn't outperform anything we have today, but the scaling path is clear. Its problems just won't matter if it does that one thing with scaling. The web is a pretty good hypermedia platform, but a disastrously bad pla…

How on earth could you evaluate the scaling path with too little information. That's my point. You can't possibly know that a technology can solve a given kind of problem if it can only so far solve a completely different kind of problem which is largely unrelated!

Saying that performance on grade-school problems is predictive of performance on complex reasoning tasks (including theorem proving) is like saying that a new kind of mechanical engine that has 90% efficiency can be scaled 10x.

These kind of scaling claims drive investment, I get it. But to someone who understands (and is actually working on) the actual problem that needs solving, this kind of claim is perfectly transparent!

Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

#566

Post by itronitron »

zx8080 wrote: post #331
zxcvgrsd2 wrote: post #9

Two Sam Altman comments that seem to be referring to this same Q* discovery. November 18 comment at APEC (just before the current drama) [1]: > On a personal note, like four times now in the history of OpenAI, the most recent time was just in the last couple of weeks , I’ve gotten to be in the room when we pushed the veil of ignorance back and the frontier of discovery forward and a September 22 Tweet [2] > sure 10x…

> > sure 10x engineers are cool but damn those 10,000x engineer/researchers... What was he referring to?

They're getting high on their own supply.

Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

#567

Post by raincole »

muskmusk wrote: post #509
wbhart wrote: post #491

I feel very comfortable saying, as a mathematician, that the ability to solve grade school maths problems would not be at all a predictor of ability to solve real mathematical problems at a research level. The reason LLMs fail at solving mathematical problems is because: 1) they are terrible at arithmetic, 2) they are terrible at algebra, but most importantly, 3) they are terrible at complex reasoning (more specifica…

Friend, the creator of this new progress is a machine learning PhD with a decade of experience in pushing machine learning forward. He knows a lot of math too. Maybe there is a chance that he too can tell the difference between a meaningless advance and an important one?

But he also has the incentive to exaggerate the AI's ability.

The whole idea of double-blind test (and really, the whole scientific methodology) is based on one simple thing: even the most experienced and informed professionals can be comfortably wrong.

We'll only know when we see it. Or at least when several independent research groups see it.

Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

#568

Post by ssnistfajen »

gizajob wrote: post #518
ssnistfajen wrote: post #467

Earlier quoted context omitted.

Occupations like computer (human form), typist, telephone switcher, all became completely eliminated when the PC came out. Jobs like travel agents are on permanent decline minus select scenarios where it is attached with luxury. Cashier went from a decent nonlaborious job to literal starvation gig because the importance of a human in the job became negligible. There are many more examples. Some people managed to retr…

Yeah but then capitalism breaks down because nobody is earning wages. One of the things capitalism is good at is providing (meaningless) employment to people because most wouldn’t know what to do with their days if given the free time back. This will only continue.

I do hope that will be the case. Certainly far better than the alternatives.

Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

#570

Post by nullc »

wbhart wrote: post #491

I feel very comfortable saying, as a mathematician, that the ability to solve grade school maths problems would not be at all a predictor of ability to solve real mathematical problems at a research level. The reason LLMs fail at solving mathematical problems is because: 1) they are terrible at arithmetic, 2) they are terrible at algebra, but most importantly, 3) they are terrible at complex reasoning (more specifica…

It's also hard to know what the LLM has reasoned out vs has memorized.

I like the very last example in my tongue-in-cheek article, https://nt4tn.net/articles/aixy.html

Certainly the LLM didn't derive Fermat's theorem on sums of two squares under the hood (and, of course, very obviously didn't prove it correct-- as the code is technically incorrect for 2), but I'm somewhat doubtful that there was any function exactly like the template in codex's training set either (at least I couldn't quickly find any published code that did that). The line between creating something and applying a memorized fact in a different context is not always super clear.

Post reply on HN