Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

reuters.com

Post reply on HN
Print view View original

371–380 of 1001 posts

Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

#371

Post by valine »

giarc wrote: post #369
huitzitziltzin wrote: post #358

Put this on my tombstone after the robots kill me or whatever, but I think all “AI safety” concerns are a wild overreaction totally out of proportion to the actual capabilities of these models. I just haven’t seen anything in the past year which makes me remotely fearful about the future of humanity, including both our continued existence and our continued employment.

Apparently what made this person fearful was grade school math. "Though only performing maths on the level of grade-school students, acing such tests made researchers very optimistic about Q*’s future success, the source said."

The response gets more reasonable the smaller the model in question. A 1B parameter model passing grade-school math tests would be much more alarming (exciting?) than a GPT-4 sized model doing the same.

GPT-4 probably has some version of the answer memorized. There’s no real explanation for a 1B parameter model solving math problems other than general cognition.

Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

#372

Post by charlie0 »

gadders wrote: post #83
gadders wrote: post #60

Weirdly enough, this sort of lines up with a theory posted on 4chan 4 days ago. The gist being that if the version is formally declared AGI, it can't be licensed to Microsoft and others for commercial gain. As a result Altman wants it not to be called AGI, other board members do. Archived link below. NB THIS IS 4CHAN - THERE WILL OFFENSIVE LANGUAGE. https://archive.ph/sFMXa

Entire text below to save the 4chan bits: part 1 There is a massive disagreement on AI safety and the definition of AGI. Microsoft invested heavily in OpenAI, but OpenAI's terms was that they could not use AGI to enrich themselves. According to OpenAI's constitution: AGI is explicitly carved out of all commercial and IP licensing agreements, including the ones with Microsoft. Sam Altman got dollar signs in his eyes w…

What's not clear here is who's call is it to declare AGI? I would think that would be Ilya. In that case, this narrative doesn't make sense.

Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

#373

Post by markisus »

CaptainNegative wrote: post #282
dougmwne wrote: post #18

Earlier quoted context omitted.

This makes some sense to me. My experience with GPT is that it is capable of straightforward logical inference, but not more inspired thinking. It lacks the ability for a “eureka moment”. All complex inference it appears to have is a result of its training set. It is incapable of solving certain kinds of logic problems that a child would be able to solve. As an example, take the wolf, goat, and cabbage problem, but c…

Here's a simpler example that works. Q: Prove that if cos(x) is an integer then so is sin(x). A: The statement "if cos(x) is an integer, then so is sin(x)" is not true in general. To understand why, let’s consider the properties of sine and cosine. [...] It had trouble when the problem domain is different from the usual one the same keywords typically fall into.

Nice one.

If cos(x) is an integer, then it is -1, 0, or 1. Then x is an integer multiple of pi/2. Then sin(x) is also -1, 0, or 1.

If ChatGPT reads this thread now it will know the answer.

Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

#374

Post by dilyevsky »

Enginerrrd wrote: post #273
TillE wrote: post #62

Earlier quoted context omitted.

GPT-4 is not solving logic problems, it's doing an extremely good job of generating text based on recognizing familiar patterns. If you give it something weird and unfamiliar, it will absolutely fail.

That's simply not true. I've invented totally novel logical problems in the form of stories for it to solve and it has done so successfully. It doesn't get it everytime in the first way that I ask but it absolutely will get with relatively modest rewriting of the prompt. In one case it finally had to ask me why I was asking it, and offered a couple of suggestions including "is this a way to test my reasoning capabili…

There’s a much simpler explanation - your “invetions” are not that original

I “invent” stuff at my dayjob all the time, then i find that at least 10 people already did that on github

Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

#375

Post by gzer0 »

NewsaHackO wrote: post #312
layer8 wrote: post #275

Earlier quoted context omitted.

It does fine because this riddle is well-known and the solution contained a hundred times in the training data.

This is just moving goalposts. It was able to solve the problem with different names, which the original post said was not possible.

Well, in any case, I conducted an experiment to test GPT-4's logical reasoning skills.

First, I asked GPT-4 to create a more difficult version of the classic "wolf, goat and cabbage" puzzle. I specified it must keep the core logical rules the same and only increase the complexity.

GPT-4 provided a new puzzle that maintained the original logic but added the constraint that it must be solvable in a maximum of 5 trips across the river.

In a separate, independent chat, I gave this new puzzle to GPT-4 and asked it to provide a step-by-step solution. It output an answer.

Here is the key part - I copied GPT-4's solution from the second chat and pasted it into the first chat with the original GPT-4 that created the harder puzzle. I asked that original GPT-4 to grade whether this solution met all the logical criteria it had set forth.

Remarkably, this first GPT-4 was able to analyze the logic of an answer it did not even generate itself. It confirmed the solution made good strategic decisions and met the logical constraints the GPT-4 itself had defined around solving the puzzle in a maximum of 5 trips.

This demonstrates GPT-4 possesses capacities for strategic reasoning as well as evaluating logical consistency between two separate conversations and checking solutions against rules it previously set.

https://chat.openai.com/share/996583dd-962b-42a8-b4b9-e29c59...

Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

#376

Post by Xelynega »

gexla wrote: post #336
maxglute wrote: post #210

Earlier quoted context omitted.

I haven't followed the situation as closely as others, but it does seem like board structured to not have 7 figure cheques that could influence / undermine safety mission seemingly willing to let openAI burn out of dogma. Employees want their 7 figure cheques, their interests aligned with deeper pockets and larger powers with 10+ figures on the line. Reporting so far have felt biased accordingly. If this was about mo…

Seems like the board's mission went off the rails long ago and it acted late. Some snippets from the OpenAI website... "Investing in OpenAI Global, LLC is a high-risk investment" "Investors could lose their capital contribution and not see any return" "It would be wise to view any investment in OpenAI Global, LLC in the spirit of a donation, with the understanding that it may be difficult to know what role money will…

Could it be possible that the board was just naive and believed that they would be able to control a capped-profit arm, but the inevitable mechanisms of capital eventually took over once they let them in?

Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

#377

Post by margorczynski »

sweezyjeezy wrote: post #212
vjerancrnjak wrote: post #81

Earlier quoted context omitted.

OpenAI already benchmarks their GPTs on leetcode problems and even includes a Codeforces rating. It is not impressive at all and there's almost no progress from GPT 2 to 4. I agree, why does this grade school math problem matter if the model can't solve problems that are very precisely stated and have a very narrow solution space (at least more narrow than some vague natural language instruction)?

Here is something that I think would be a big breakthrough: I explain to GPT in text, a mathematical concept it has never seen in its training data and give a few examples (not inferred from fill the blank on millions of examples). It actually learns this to update its weights - not just uses it as part of a prompt. Extrapolating this optimistically - this is a huge step towards AGI in my opinion. You can (in theory)…

> It actually learns this to update its weights - not just uses it as part of a prompt.

In a way both are the same thing - memory that is in a feedback loop with the network that does the calculation. Just that the weights give much faster access, no "serde".

Maybe the goal is not to modify the weights but train the network so that it can effectively use a "memory block" in the way it works. Now this is in a way faked by re-feeding the output it produces concatenated with the original phrase. Don't we as humans effectively extend our memory by using all kind of text, written or digital? Just the issue is that it is slow to utilize, for a computer using fast RAM that wouldn't be much of an issue.

Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

#378

Post by confused_boner »

CaptainNegative wrote: post #282
dougmwne wrote: post #18

Earlier quoted context omitted.

This makes some sense to me. My experience with GPT is that it is capable of straightforward logical inference, but not more inspired thinking. It lacks the ability for a “eureka moment”. All complex inference it appears to have is a result of its training set. It is incapable of solving certain kinds of logic problems that a child would be able to solve. As an example, take the wolf, goat, and cabbage problem, but c…

Here's a simpler example that works. Q: Prove that if cos(x) is an integer then so is sin(x). A: The statement "if cos(x) is an integer, then so is sin(x)" is not true in general. To understand why, let’s consider the properties of sine and cosine. [...] It had trouble when the problem domain is different from the usual one the same keywords typically fall into.

https://chat.openai.com/share/cc056ef8-f2c3-4fff-ba60-13b53f...

My Custom Instructions: https://lensdump.com/i/sg3q6C

Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

#379

Post by JoshTko »

gadders wrote: post #83
gadders wrote: post #60

Weirdly enough, this sort of lines up with a theory posted on 4chan 4 days ago. The gist being that if the version is formally declared AGI, it can't be licensed to Microsoft and others for commercial gain. As a result Altman wants it not to be called AGI, other board members do. Archived link below. NB THIS IS 4CHAN - THERE WILL OFFENSIVE LANGUAGE. https://archive.ph/sFMXa

Entire text below to save the 4chan bits: part 1 There is a massive disagreement on AI safety and the definition of AGI. Microsoft invested heavily in OpenAI, but OpenAI's terms was that they could not use AGI to enrich themselves. According to OpenAI's constitution: AGI is explicitly carved out of all commercial and IP licensing agreements, including the ones with Microsoft. Sam Altman got dollar signs in his eyes w…

This is the first explanation that is consistent with all participant motivations

Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

#380

Post by jhbadger »

huitzitziltzin wrote: post #358

Put this on my tombstone after the robots kill me or whatever, but I think all “AI safety” concerns are a wild overreaction totally out of proportion to the actual capabilities of these models. I just haven’t seen anything in the past year which makes me remotely fearful about the future of humanity, including both our continued existence and our continued employment.

Exactly. The rational fear is that they will automate many lower middle class jobs and cause unemployment, not that Terminator was a documentary.
Post reply on HN