Live data from Hacker News

OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

reuters.com

81–90 of 1001 posts

Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

#81
post #3

> warning of a powerful artificial intelligence discovery that they said could threaten humanity, > Given vast computing resources, the new model was able to solve certain mathematical problems, [..] Though only performing math on the level of grade-school students, acing such tests made researchers very optimistic about Q*’s future success, the source said. I somehow expected a bit more.

OpenAI already benchmarks their GPTs on leetcode problems and even includes a Codeforces rating. It is not impressive at all and there's almost no progress from GPT 2 to 4.

I agree, why does this grade school math problem matter if the model can't solve problems that are very precisely stated and have a very narrow solution space (at least more narrow than some vague natural language instruction)?

Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

#82

Earlier quoted context omitted.

GPT4 solves this problem In any combination easily. What do so many posters seem to claim to have stumped it?

Because I did stump it, not claim to. Used GPT-4 near when it was released, fed it variations on the problem for about an hour. I found it to be a fascinating exercise because it was teaching be the difference between knowledge (training set) and intelligence (generalization).

It has overfit to the exact wolf, cabbage and goat example but literally change the names to whatever you want and it consistently solves it just fine.

Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

#83
post #60

Weirdly enough, this sort of lines up with a theory posted on 4chan 4 days ago. The gist being that if the version is formally declared AGI, it can't be licensed to Microsoft and others for commercial gain. As a result Altman wants it not to be called AGI, other board members do. Archived link below. NB THIS IS 4CHAN - THERE WILL OFFENSIVE LANGUAGE. https://archive.ph/sFMXa

Entire text below to save the 4chan bits:

part 1 There is a massive disagreement on AI safety and the definition of AGI. Microsoft invested heavily in OpenAI, but OpenAI's terms was that they could not use AGI to enrich themselves. According to OpenAI's constitution: AGI is explicitly carved out of all commercial and IP licensing agreements, including the ones with Microsoft. Sam Altman got dollar signs in his eyes when he realized that current AI, even the proto-AGI of the present, could be used to allow for incredible quarterly reports and massive enrichment for the company, which would bring even greater investment. Hence Dev Day. Hence the GPT Store and revenue sharing. This crossed a line with the OAI board of directors, as at least some of them still believed in the original ideal that AGI had to be used for the betterment of mankind, and that the investment from Microsoft was more of a "sell your soul to fight the Devil" sort of a deal. More pragmatically, it ran the risk of deploying deeply "unsafe" models. Now what can be called AGI is not clear cut. So if some major breakthrough is achieved (eg Sam saying he recently saw the veil of ignorance being pushed back), can this breakthrough be called AGI depends on who can get more votes in the board meeting. And if one side can get enough votes to declare it AGI, Microsoft and OpenAI could loose out billions in potential licence agreements. And if one side can get enough votes to declare it not AGI, then they can licence this AGI-like tech for higher profits.

Few weeks/months ago OpenAI engineers made a breakthrough and something resembling AGI was achieved (hence his joke comment, the leaks, vibe change etc). But Sam and Brockman hid the extent of this from the rest of the non-employee members of the board. Ilyas is not happy about this and feels it should be considered AGI and hence not licensed to anyone including Microsoft. Voting on AGI status comes to the board, they are enraged about being kept in the dark. They kick Sam out and force Brockman to step down. Ilyas recently claimed that current architecture is enough to reach AGI, while Sam has been saying new breakthroughs are needed. So in the context of our conjecture Sam would be on the side trying to monetize AGI and Ilyas will be the one to accept we have achieved AGI. Sam Altman wants to hold off on calling this AGI because the longer it's put off, the greater the revenue potential. Ilya wants this to be declared AGI as soon as possible, so that it can only be utilized for the company's original principles rather than profiteering. Ilya winds up winning this power struggle. In fact, it's done before Microsoft can intervene, as they've declared they had no idea that this was happening, and Microsoft certainly would have incentive to delay the declaration of AGI. Declaring AGI sooner means a combination of a lack of ability for it to be licensed out to anyone (so any profits that come from its deployment are almost intrinsically going to be more societally equitable and force researchers to focus on alignment and safety as a result) as well as regulation. Imagine the news story breaking on /r/WorldNews: "Artificial General Intelligence has been invented." And it spreads throughout the grapevine the world over, inciting extreme fear in people and causing world governments to hold emergency meetings to make sure it doesn't go Skynet on us, meetings that the Safety crowd are more than willing to have held.

part 3 This would not have been undertaken otherwise. Instead, we'd push forth with the current frontier models and agent sharing scheme without it being declared AGI, and OAI and Microsoft stand to profit greatly from it as a result, and for the Safety crowd, that means less regulated development of AGI, obscured by Californian principles being imbued into ChatGPT's and DALL-E's outputs so OAI can say "We do care about safety!" It likely wasn't Ilya's intention to ouster Sam, but when the revenue sharing idea was pushed and Sam argued that the tech OAI has isn't AGI or anything close, that's likely what got him to decide on this coup. The current intention by OpenAI might be to declare they have an AGI very soon, possibly within the next 6 to 8 months, maybe with the deployment of GPT-4.5 or an earlier-than-expected release of 5. Maybe even sooner than that. This would not be due to any sort of breakthrough; it's using tech they already have. It's just a disagreement-turned-conflagration over whether or not to call this AGI for profit's sake.

Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

#84
post #61

Earlier quoted context omitted.

Mathematical logic doesn’t refer to ‘complex word problems with big numbers’. The reason that LLMs struggle (well, are hopeless currently) with mathematics problems is not because it involves arithmetic (although this is already enough of a struggle) but because it requires a conceptual understanding of the ideas rather than just being about manipulating language — a lot of the goings-on are hidden in the semantic wo…

I've seen it gets tripped up if you mix something common with something uncommon. ChatGPT chokes on, for example: "Two trains on separate tracks, 30 miles from each other are approaching each other, each at a speed of 10 mph. How long before they crash into each other?"

Answer from GPT-4 Turbo: If two trains are 30 miles apart and are approaching each other at a speed of 10 mph each, their combined approach speed is 20 mph (10 mph + 10 mph). To find the time it takes for them to meet, you divide the distance by their combined speed: 30 miles / 20 mph = 1.5 hours Therefore, it will take 1.5 hours before they meet (or crash into each other, as stated in the question). Two trains on separate tracks, 30 miles from each other are approaching each other, each at a speed of 10 mph. How long before they crash into each other?

Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

#85
post #60

Weirdly enough, this sort of lines up with a theory posted on 4chan 4 days ago. The gist being that if the version is formally declared AGI, it can't be licensed to Microsoft and others for commercial gain. As a result Altman wants it not to be called AGI, other board members do. Archived link below. NB THIS IS 4CHAN - THERE WILL OFFENSIVE LANGUAGE. https://archive.ph/sFMXa

Well it isn't AGI - so it sounds like the board is more interested in keeping MSFT from making money than whether or not it's actually AGI.

Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

#86
post #41
post #20

This sort is why limits to private wealth and power are, IMO, a good idea We have already seen petroleum companies burn the world for profit Tech is doing its best to eliminate privacy for profit What would Sam do or not do, for profit?

I don’t understand this sentiment. Aren’t consumers doing the burning?

I'd like to have the choice not to. I have extremely limited options for that.

And I already live in a country which regularly has all of its electricity generated renewably.

Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

#87

Earlier quoted context omitted.

Because I did stump it, not claim to. Used GPT-4 near when it was released, fed it variations on the problem for about an hour. I found it to be a fascinating exercise because it was teaching be the difference between knowledge (training set) and intelligence (generalization).

http://allofus.ai/journeys/735TTpCUncpAQC9yUqM3ve Easily solved it everytime On allofus.ai using basic reflection and mixture of experts.

It says right up front that it recognized the problem formulation from its training set. You need to change the context and formulation enough that it’s no longer able to parrot back the “classic solution” and actually has to do its own logical inference.

Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

#89

Why does OpenAI get to define what AGI is?

You don't need to accept their definition, neither does anyone else. But they do need to have a definition that they accept themselves, because it's used throughout their charter.

https://openai.com/charter

Post reply on HN