Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

reuters.com

Post reply on HN
Print view View original

21–30 of 1001 posts

Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

#21

Post by fullshark »

V__ wrote: post #3

> warning of a powerful artificial intelligence discovery that they said could threaten humanity, > Given vast computing resources, the new model was able to solve certain mathematical problems, [..] Though only performing math on the level of grade-school students, acing such tests made researchers very optimistic about Q*’s future success, the source said. I somehow expected a bit more.

My guess is if you see AGI as inevitable, then every step towards it is seen as groundbreaking with unprecedented future impact on humanity.

Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

#22

Post by LewisVerstappen »

V__ wrote: post #3

> warning of a powerful artificial intelligence discovery that they said could threaten humanity, > Given vast computing resources, the new model was able to solve certain mathematical problems, [..] Though only performing math on the level of grade-school students, acing such tests made researchers very optimistic about Q*’s future success, the source said. I somehow expected a bit more.

The entire criticism of LLMs is that they're amazing at recall and bad at reasoning.

The discovery OpenAI could've made might've been on techniques to teach LLMs reasoning.

That would be very big.

Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

#23

Post by ilaksh »

bitshiftfaced wrote: post #15
ilaksh wrote: post #12

Could it be named in reference to something like the A* search algorithm? What if Q stands for Query in transformer attention? How would that type of search translate to transformers? They said that it's able to solve simple math problems. If it's related to A* then maybe it's trying to find a path to something. The answer to a word problem?

My mind went to Q learning.

Ok so maybe nothing to do with A*, but actually a way for GPT-powered models or agents to learn through automated reinforcement learning. Or something.

I wonder if DeepMind is working on something similar also.

If your hunch is right, this could lead to the type of self-improvement that scares people.

Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

#25

Post by willsmith72 »

I still don't really know how they're measuring AGI. They all seem very sure about it, does someone have a good definition?

OpenAI just say "AI systems that are generally smarter than humans". That's not measurable.

What will it look like when AGI is achieved?

Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

#27

Post by _lvbh »

LewisVerstappen wrote: post #16
htk wrote: post #6

Just when I thought the ride was over. The belief in what's in the letter could explain some things like how the board couldn't trust Sam to keep this "discovery" at bay, and how it could be better to implode the company than let it explore said technology.

> the board couldn't trust Sam to keep this "discovery" at bay, and how it could be better to implode the company than let it explore said technology. Half the board members have a background in ML. Why should they be able to make the decision to implode the entire company over this? Why should they have the ability to give zero transparency or comms to the public on their decision? OpenAI's governance structure was…

> None of the board members have a background in ML.

The same can be said for Altman. Ilya at least has a research background in ML. The rest, I don’t know

Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

#28

Post by imdsm »

dougmwne wrote: post #18
V__ wrote: post #3

> warning of a powerful artificial intelligence discovery that they said could threaten humanity, > Given vast computing resources, the new model was able to solve certain mathematical problems, [..] Though only performing math on the level of grade-school students, acing such tests made researchers very optimistic about Q*’s future success, the source said. I somehow expected a bit more.

This makes some sense to me. My experience with GPT is that it is capable of straightforward logical inference, but not more inspired thinking. It lacks the ability for a “eureka moment”. All complex inference it appears to have is a result of its training set. It is incapable of solving certain kinds of logic problems that a child would be able to solve. As an example, take the wolf, goat, and cabbage problem, but c…

And thus the divide between decels and accels came to the fore.

The decels wanted to destroy OpenAI (the mission) to stop progress on Q*.

This is like a jigsaw but bit by bit, the picture is coming together.

Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

#29

Post by loeber »

LewisVerstappen wrote: post #16
htk wrote: post #6

Just when I thought the ride was over. The belief in what's in the letter could explain some things like how the board couldn't trust Sam to keep this "discovery" at bay, and how it could be better to implode the company than let it explore said technology.

> the board couldn't trust Sam to keep this "discovery" at bay, and how it could be better to implode the company than let it explore said technology. Half the board members have a background in ML. Why should they be able to make the decision to implode the entire company over this? Why should they have the ability to give zero transparency or comms to the public on their decision? OpenAI's governance structure was…

Sorry, are you saying that Ilya Sutskever doesn't have an ML background? That Adam D'Angelo can't code? Come on.

Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

#30

Post by flibble »

bitshiftfaced wrote: post #15
ilaksh wrote: post #12

Could it be named in reference to something like the A* search algorithm? What if Q stands for Query in transformer attention? How would that type of search translate to transformers? They said that it's able to solve simple math problems. If it's related to A* then maybe it's trying to find a path to something. The answer to a word problem?

My mind went to Q learning.

Mine to Q of Star Trek.
Post reply on HN