Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

reuters.com

Post reply on HN
Print view View original

361–370 of 1001 posts

Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

#361

Post by llelouch »

zx8080 wrote: post #331
zxcvgrsd2 wrote: post #9

Two Sam Altman comments that seem to be referring to this same Q* discovery. November 18 comment at APEC (just before the current drama) [1]: > On a personal note, like four times now in the history of OpenAI, the most recent time was just in the last couple of weeks , I’ve gotten to be in the room when we pushed the veil of ignorance back and the frontier of discovery forward and a September 22 Tweet [2] > sure 10x…

> > sure 10x engineers are cool but damn those 10,000x engineer/researchers... What was he referring to?

Probably Ilya and his team. The recent discovery was made by him and his team. "The technical breakthrough, spearheaded by OpenAI chief scientist Ilya Sutskever"

Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

#362

Post by gryfft »

_heimdall wrote: post #356

> OpenAI defines AGI as autonomous systems that surpass humans in most economically valuable tasks. Are we really defining intelligence as economic value at this point? This is completely ridiculous. We have yet to decide exactly what human intelligence, how it manifests in the body, or how to reliably measure it. I get that people want to justify developing artificial intelligence before understanding intelligence i…

I'm as left leaning as HN commenters get, I think, and in terms of quacking like a duck, "surpassing humans in most economically valuable tasks" is 100% the meaningful Turing test in a capitalist society.

Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

#364

Post by [deleted] »

wegfawefgawefg wrote: post #343

If I had to guess, the name Q* is pronounced Q Star, and probably the Q refers to Q values or estimated rewards from reinforcement learning, and the star refers to a search and prune algorithm, like A* (A star). Possibly they combined deep reinforcement learning with self training and search and got a bot that could learn without needing to ingest the whole internet. Usually DRL agents are good at playing games, but…

[deleted]

Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

#365

Post by dinobones »

wegfawefgawefg wrote: post #343

If I had to guess, the name Q* is pronounced Q Star, and probably the Q refers to Q values or estimated rewards from reinforcement learning, and the star refers to a search and prune algorithm, like A* (A star). Possibly they combined deep reinforcement learning with self training and search and got a bot that could learn without needing to ingest the whole internet. Usually DRL agents are good at playing games, but…

Paper? You mean more like the Bing Search plugin?

Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

#366

Post by stygiansonic »

wegfawefgawefg wrote: post #343

If I had to guess, the name Q* is pronounced Q Star, and probably the Q refers to Q values or estimated rewards from reinforcement learning, and the star refers to a search and prune algorithm, like A* (A star). Possibly they combined deep reinforcement learning with self training and search and got a bot that could learn without needing to ingest the whole internet. Usually DRL agents are good at playing games, but…

This assumes that they will publish a paper that has substantive details about Q*

Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

#367

Post by hutzlibu »

ChuckMcM wrote: post #39
toomuchtodo wrote: post #10

Earlier quoted context omitted.

I trust AGI more than any of the humans so far who have tried to argue they can wrangle it.

I would be curious as to the basis for that trust. I struggle to find any reason that AGI would care about "humans" at all, other than during the short period of time it needed them to be cooperative actuators for machinery that is creating a replacement for humans that the AGI can control directly. My expectation for the chain of events goes something like: 1) AGI gains sentience 2) AGI "breaks out" of its original…

"other than during the short period of time it needed them to be cooperative actuators for machinery that is creating a replacement for humans that the AGI can control directly"

It would require a fully automated, self replicating industry fo a AGI to sustain itself. We are quite far from that.

" the same reason you don't care that an antibiotic kills all the bacteria in your gut"

And I do care, because it messes with my digestion, which is why I only antibiotics in very rare cases.

So far I am not convinced that AGI is possible at all with out current tech. And if it turns out it is, why should it turn out to be a godlike, selfish but emotionless being? If it has no emotions, why would it want anything, like its prolonging of existence?

Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

#368

Post by tnecniv »

brutusborn wrote: post #342

Baseless speculation: they started testing an approach similar to VERSES AI and realized it has much more potential than LLMs. The free energy principle, knowledge graph approach just seems more likely to develop general intelligence without hallucinations imo.

Eh the free energy principle isn’t that magical and that comes from someone who likes it quite a bit. At the end of the day extremizing the free energy is a relaxation of constrained optimization. There are ways it’s used I find very appealing but not so appealing I’d expect it to become the dominant paradigm. Moreover, you can write a lot of standard ML problems free energy optimization in one way or another, but I don’t think standard ML will quite get us there. We need at least one more conceptual step forward.

Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

#369

Post by giarc »

huitzitziltzin wrote: post #358

Put this on my tombstone after the robots kill me or whatever, but I think all “AI safety” concerns are a wild overreaction totally out of proportion to the actual capabilities of these models. I just haven’t seen anything in the past year which makes me remotely fearful about the future of humanity, including both our continued existence and our continued employment.

Apparently what made this person fearful was grade school math.

"Though only performing maths on the level of grade-school students, acing such tests made researchers very optimistic about Q*’s future success, the source said."

Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

#370

Post by coffeebeqn »

TheOtherHobbes wrote: post #104
ChuckMcM wrote: post #39

Earlier quoted context omitted.

I would be curious as to the basis for that trust. I struggle to find any reason that AGI would care about "humans" at all, other than during the short period of time it needed them to be cooperative actuators for machinery that is creating a replacement for humans that the AGI can control directly. My expectation for the chain of events goes something like: 1) AGI gains sentience 2) AGI "breaks out" of its original…

None of those necessarily follow. Gaining sentience is not the same as gaining infallible super-sentience. There may even be some kind of asymptotic limit on how correct and reliable sentience can be. The more general the sentience, the more likely it is to make mistakes. Maybe. Personally in the short term I'm more worried about abuse of what have already, or might credibly have in the very near future.

Exponential growth is also certainly not a given. I’m much less worried about a AGI ruling the universe in 5 seconds after it’s birth than a really good AI that causes mass unemployment
Post reply on HN