Live data from Hacker News

OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

reuters.com

211–220 of 1001 posts

Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

#211

Earlier quoted context omitted.

Mathematical logic doesn’t refer to ‘complex word problems with big numbers’. The reason that LLMs struggle (well, are hopeless currently) with mathematics problems is not because it involves arithmetic (although this is already enough of a struggle) but because it requires a conceptual understanding of the ideas rather than just being about manipulating language — a lot of the goings-on are hidden in the semantic wo…

Confusing to me actually. Are you saying coding doesn't require understanding ideas?

Not at all; just that the ‘understanding’-related content of pure mathematics is much less evident on the page. It mostly lives in the heads of mathematicians and is largely independent of notation, whereas writing blocks of code is a task that is literally about using notation — something for which a huge amount of high-quality training data exists. Of course, the higher level ideas in programming and software development are not just about ‘writing code’ itself, but I suspect this is where current models begin to show their weakness.

Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

#212
post #3

> warning of a powerful artificial intelligence discovery that they said could threaten humanity, > Given vast computing resources, the new model was able to solve certain mathematical problems, [..] Though only performing math on the level of grade-school students, acing such tests made researchers very optimistic about Q*’s future success, the source said. I somehow expected a bit more.

OpenAI already benchmarks their GPTs on leetcode problems and even includes a Codeforces rating. It is not impressive at all and there's almost no progress from GPT 2 to 4. I agree, why does this grade school math problem matter if the model can't solve problems that are very precisely stated and have a very narrow solution space (at least more narrow than some vague natural language instruction)?

Here is something that I think would be a big breakthrough:

I explain to GPT in text, a mathematical concept it has never seen in its training data and give a few examples (not inferred from fill the blank on millions of examples). It actually learns this to update its weights - not just uses it as part of a prompt.

Extrapolating this optimistically - this is a huge step towards AGI in my opinion. You can (in theory) teach it to automate many tasks, correct it's mistakes without needing costly extra training data, and move towards the few-shot (and persistent) learning that separates humans from AI right now.

Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

#213

Earlier quoted context omitted.

Makes me wonder if they stumbled onto some emergent behavior with the new Assistants API. You can have an Assistant thread spawn other Assistant threads, each with their own special instructions, plus the ability to execute custom code, reach out to the internet for other data and processing as needed, etc. Basically kicking off a hive mind that overcomes the limitations of a single LLM.

okay, i could be convinced... but what is the compute for this? you can't just "spawn threads" with reckless abandon without considering the resource requirements

As long as your checks clear and the HVAC in the data center holds up I think you're good to go.

The beauty of the Assistants is you're not limited to OpenAI models. You can wire them up to any model anywhere (out they can wire themselves up), so you can have specialist threads going for specific functions.

Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

#214
post #5

Things are gonna be lit about 30 seconds after AGI is created. Gotta practice being "Goodlife" early in case that happens. :-)

I trust AGI more than any of the humans so far who have tried to argue they can wrangle it.

Benevolent Gods do not exist. Never have, never will. So don't place your faith in them.

Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

#215
post #195

Earlier quoted context omitted.

It doesn't read like the usual "redpill me on the earth being flat" type conspiracy theories. It claims to be from an Open AI insider. I'm not saying it's true, but it does sound plausible.

This is exactly how confirmation bias fuels conspiracy theories. No one believes anything that they think sounds implausible. As a general rule, you should give very little thought to anonymous 4chan posts.

That's absolutely true.

But

They have leaked real things in the past, in exactly the same way. It may be 5% or less that turn out to be true, but there's the rub. That's why no one can completely dismiss it out of hand (and why were even discussing it on an HN comment thread in the first place).

Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

#216
>several staff researchers sent the board of directors a letter warning of a powerful artificial intelligence discovery that they said could threaten humanity

>The maker of ChatGPT had made progress on Q*, which some internally believe could be a breakthrough in the startup's search for superintelligence, also known as artificial general intelligence (AGI)

>Given vast computing resources, the new model was able to solve certain mathematical problems....Though only performing math on the level of grade-school students

I hate to agree with Elon Musk on anything, but I think he was right when he called this a nice marketing stunt this morning. It has major "Snoop Dogg giving up the smoke" vibes.

Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

#217

Remarkable that this unattributed claim of a true AGI breakthrough comes with a name that's impossible to use as a search term. I'm not saying, I'm just saying.

Who is Q?

I mean, I'll take this seriously when there's something more substantive than zero meaningful search results and a /pol/ post to evaluate it against.

Right now it reads like something a moderately clever shitposter would invent - if there was a letter, why not just leak the letter? - and while everyone's clearly very excited over the events of the past week, I'd like to hope epistemic personal hygiene has not been entirely abandoned.

Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

#218
post #62

Earlier quoted context omitted.

GPT-4 is not solving logic problems, it's doing an extremely good job of generating text based on recognizing familiar patterns. If you give it something weird and unfamiliar, it will absolutely fail.

> solving logic problems > generating text based on recognizing familiar patterns That's the same thing.

Here is me asking ChatGPT the classic simple wolf/sheep/cabbage logic question, but changing things so that the sheep eats the wolf, the wolf eats the rutabega, and the rutabega eats the sheep.

ChatGPT immediately screws it up because it isn't actually solving the logic problem, it's just pattern-matching, and the strong pattern is for the wolf to eat the sheep.

https://chat.openai.com/share/e9b88c45-e69b-4da4-b423-004d07...

EDIT: As noted in a subthread, I outclevered myself and also a bunch of human commentators, as the problem I proposed doesn't have a solution and neither ChatGPT, I, and also other presumably human commentators correctly identified that -- I had everything eat something, and the normal wolf/sheep/cabbage of course has the cabbage eating nothing.

When I tried again with sheep-eats-wolf, wolf-eats-rutabaga, rutabaga-eats-nothing, it correctly solved the problem on the first try.

https://chat.openai.com/share/5a2700de-1850-4f25-8adf-2d2b97...

Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

#219
post #3

> warning of a powerful artificial intelligence discovery that they said could threaten humanity, > Given vast computing resources, the new model was able to solve certain mathematical problems, [..] Though only performing math on the level of grade-school students, acing such tests made researchers very optimistic about Q*’s future success, the source said. I somehow expected a bit more.

I think there is enough there to make me believe it's real. After all, a big objection to LLMs is they are just "fancy autocomplete" or "stochastic parrots". I don't agree, but LLMs famously are "bad at math", that is you can't really train them to do math unless they've seen previous calculations with similar numbers in their training sets. Training an AI system to truly do arbitrary mathematical logic (think "compl…

One more interesting thing.

Here is how GPT-4 performs long multiplication: https://www.phind.com/search?q=multiply%2012.23341%20by%20-1...

Here is how GPT-$ Turbo performs long multiplication: multiply 12.23341 by -123.23354 using long multiplication method. To multiply 12.23341 by -123.23354 using the long multiplication method, we first ignore the decimal points and treat the numbers as integers. We'll reintroduce the decimal point at the end. Here's the multiplication step by step: ``` 1223341 (this is 12.23341 with the decimal removed) × -12323354 (this is -123.23354 with the decimal removed) -------------- 1223341 (multiplied by 4) 0000000 (multiplied by 5, shifted one position to the left) 1223341 (multiplied by 3, shifted two positions to the left) 3660023 (multiplied by 3, shifted three positions to the left) 4886682 (multiplied by 2, shifted four positions to the left) 0000000 (multiplied by 2, shifted five positions to the left) 1223341 (multiplied by 1, shifted six positions to the left) 1223341 (multiplied by 1, shifted seven positions to the left) 0000000 (multiplied by 2, shifted eight positions to the left) 0000000 (multiplied by 3, shifted nine positions to the left) -------------- 15077944092654 (sum of all the above products) ``` Now, we need to place the decimal point. The original numbers had 5 and 8 decimal places, respectively, so the result should have 5+8 = 13 decimal places. So, 15077944092654 becomes 1507.7944092654. Since one of the original numbers was negative, the result is also negative: -1507.7944092654

Therefore, 12.23341 multiplied by -123.23354 equals -1507.7944092654.

Compare with correct result: -1507.5664205714

Here is how I experimented with long multiplication on GPT-4 a month ago (5 shot, GPT-4 made mistakes on addition in 3rd some time 4th digit, regardless of decimal point position):

Multiply -0.9923 by -0.9923 using long multiplication.Solution: multiply individual digits in reverse order, increasing the order of magnitude of the result for each next digit in the first number : 3310*(0+0)=9, 3210*(0+1)=60, 3910*(0+2)=2700, 3910*(0+3)=27000, 3010*(0+4)=0, step's subtotal[0]=29769; 2310*(1+0)=60, 2210*(1+1)=400, 2910*(1+2)=18000, 2910*(1+3)=180000, 2010*(1+4)=0, step's subtotal[1]=198460; 9310*(2+0)=2700, 9210*(2+1)=18000, 9910*(2+2)=810000, 9910*(2+3)=8100000, 9010*(2+4)=0, step's subtotal[2]=8930700; 9310*(3+0)=27000, 9210*(3+1)=180000, 9910*(3+2)=8100000, 9910*(3+3)=81000000, 9010*(3+4)=0, step's subtotal[3]=89307000; 0310*(4+0)=0, 0210*(4+1)=0, 0910*(4+2)=0, 0910*(4+3)=0, 0010*(4+4)=0, step's subtotal[4]=0; Sum of partial results: 29769+198460+8930700+89307000+0 = 98465929. Set the decimal point position in the result by adding the decimal places of both numbers (4+4=8), counting from the right. Final result: -0.9923*-0.9923=0.98465929

I was able to tune the LLaMA 2 with QLoRA to produce viable results only with precision up to 4th digit after decimal point, however difference in length of mantissa cases wrong result.

Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

#220
post #114

Earlier quoted context omitted.

None of us have seen the letter so this may be off base, but I would expect people working at the world's most prominent AI research organization to have more skepticism about the ramifications of any one "breakthrough." Perhaps most did, but a couple didn't and wrote the letter? More than 3 decades ago when AI started beating humans at chess, some people feared AGI was right around the corner. They were wrong. Last…

>Last year a Google researcher thought his chat bot was sentient ( https://www.scientificamerican.com/article/google-engineer-c ... .) He was wrong. You've figured out how to test for sentience ?

[deleted]
Post reply on HN