Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

reuters.com

Post reply on HN
Print view View original

161–170 of 1001 posts

Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

#161

Post by eli_gottlieb »

golergka wrote: post #91
gadders wrote: post #60

Weirdly enough, this sort of lines up with a theory posted on 4chan 4 days ago. The gist being that if the version is formally declared AGI, it can't be licensed to Microsoft and others for commercial gain. As a result Altman wants it not to be called AGI, other board members do. Archived link below. NB THIS IS 4CHAN - THERE WILL OFFENSIVE LANGUAGE. https://archive.ph/sFMXa

There's been rumours of AI discovering new physics on twitter as well as here on HN. "Solving equations" could mean the same thing.

SINDy does that. https://github.com/dynamicslab/pysindy

Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

#162

Post by rossdavidh »

"Given vast computing resources, the new model was able to solve certain mathematical problems, the person said on condition of anonymity because they were not authorized to speak on behalf of the company. Though only performing math on the level of grade-school students, acing such tests made researchers very optimistic about Q*’s future success, the source said."

So, uh, given vast computing resources, they were able to solve math problems on the level of grade-school students? Uh-huh. I am not, at this point, impressed.

Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

#163

Post by cactusplant7374 »

lucubratory wrote: post #126

Well, Emmett Shear lied to everyone if he knew about this. I understand why, he was probably thinking that without any ability to actually undo it the best that could be done would be to make sure that no one else knows about it so that it doesn't start an arms race, but we all know now. Given the Board's silence and inadequate explanations, they may have had the same reasoning. Mira evidently didn't have the same co…

It tells us exactly the capabilities of Q.

> Given vast computing resources, the new model was able to solve certain mathematical problems, the person said on condition of anonymity because they were not authorized to speak on behalf of the company. Though only performing math on the level of grade-school students, acing such tests made researchers very optimistic about Q’s future success, the source said.

Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

#164

Post by reasonableklout »

V__ wrote: post #3

> warning of a powerful artificial intelligence discovery that they said could threaten humanity, > Given vast computing resources, the new model was able to solve certain mathematical problems, [..] Though only performing math on the level of grade-school students, acing such tests made researchers very optimistic about Q*’s future success, the source said. I somehow expected a bit more.

I don't really understand this. Aren't LLMs already performing at near-expert level on "certain mathematical problem" benchmarks?

For example, over a year ago MINERVA from Google [1] got >50% on the MATH dataset, a set of competition math problems. These are not easy problems. From the MATH dataset paper:

> We also evaluated humans on MATH, and found that a computer science PhD student who does not especially like mathematics attained approximately 40% on MATH, while a three-time IMO gold medalist attained 90%, indicating that MATH can be challenging for humans as well.

[1] https://blog.research.google/2022/06/minerva-solving-quantit...

Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

#165

Post by hn_throwaway_99 »

pk-protect-ai wrote: post #78
hn_throwaway_99 wrote: post #17

Earlier quoted context omitted.

I think there is enough there to make me believe it's real. After all, a big objection to LLMs is they are just "fancy autocomplete" or "stochastic parrots". I don't agree, but LLMs famously are "bad at math", that is you can't really train them to do math unless they've seen previous calculations with similar numbers in their training sets. Training an AI system to truly do arbitrary mathematical logic (think "compl…

I compared GPT-4 Turbo with my previous tests on GPT-4, and the results are quite interesting. GPT-4 Turbo is better at arithmetic and makes fewer errors in multiplying four-digit numbers. In fact, it makes significantly fewer errors with five-digit numbers. The level of errors on five-digit numbers is high but much lower than with four-digit numbers in GPT-4. multiplication of floats XX.MMMM and YYY.ZZZZ produces er…

But the point about how it just "improves" with slightly larger numbers, but still fails at really big numbers, shows that it's not really "reasoning" about math in a logical way - that's the point I was getting at.

For example, once you teach a grade schooler the basic process for addition, they can add 2 30 digit numbers correctly fairly easily (whether they want to do it or not is a different story). The fact that LLMs still make errors at larger numbers points to the fact that they're not really "learning" the rules of arithmetic.

Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

#167

Post by eli_gottlieb »

bitshiftfaced wrote: post #15
ilaksh wrote: post #12

Could it be named in reference to something like the A* search algorithm? What if Q stands for Query in transformer attention? How would that type of search translate to transformers? They said that it's able to solve simple math problems. If it's related to A* then maybe it's trying to find a path to something. The answer to a word problem?

My mind went to Q learning.

My mind went to some kind of Q-learning combined with something like a Monte Carlo Tree Search with some kind of A*-style heuristic to effectively combine Q-learning and with short-horizon planning.

Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

#168

Post by i_call_solo »

golergka wrote: post #101
TillE wrote: post #62

Earlier quoted context omitted.

GPT-4 is not solving logic problems, it's doing an extremely good job of generating text based on recognizing familiar patterns. If you give it something weird and unfamiliar, it will absolutely fail.

> solving logic problems > generating text based on recognizing familiar patterns That's the same thing.

Yes, but it's only able to solve problems that humans have already solved. The Eureka moment is when it starts solving problems that humans have not.

Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

#169

Post by waterhouse »

slibhb wrote: post #140
ChuckMcM wrote: post #39

Earlier quoted context omitted.

I would be curious as to the basis for that trust. I struggle to find any reason that AGI would care about "humans" at all, other than during the short period of time it needed them to be cooperative actuators for machinery that is creating a replacement for humans that the AGI can control directly. My expectation for the chain of events goes something like: 1) AGI gains sentience 2) AGI "breaks out" of its original…

Why would AGI prefer want to avoid being turned off? Why would it want to spread across the universe? Those seem like petty human concerns derived from our evolutionary history. I see no reason to assume a superintelligent AI would share them.

If it has any goals at all, then being turned off is likely to prevent it from accomplishing those goals, whereas conquering its local environment is likely to help it achieve those goals, and spreading across the universe would help with a broad subset of possible goals. "Instrumental convergence" is the search term here.

Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

#170

Post by MikeAmelung »

ALittleLight wrote: post #100
dougmwne wrote: post #58

Earlier quoted context omitted.

Because I did stump it, not claim to. Used GPT-4 near when it was released, fed it variations on the problem for about an hour. I found it to be a fascinating exercise because it was teaching be the difference between knowledge (training set) and intelligence (generalization).

Solved it first attempt for me - https://chat.openai.com/share/d60f492b-cfd6-4c08-91b9-fbd767...

Not really, because you said they were on the East and you wanted to move them to the East. Correct answer would have been "Don't do anything". If this is how they're getting AI to "follow instructions", we're completely fucked.
Post reply on HN