Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

reuters.com

Post reply on HN
Print view View original

551–560 of 1001 posts

Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

#551

Post by sampo »

synaesthesisx wrote: post #151

Remember, about a month ago Sam posted a comment along the lines of "AI will be capable of superhuman persuasion well before it is superhuman at general intelligence, which may lead to very strange outcomes". The board was likely spooked by the recent breakthroughs (which were most likely achieved by combining transformers with another approach), and hit the panic button. Anything capable of "superhuman persuasion",…

> Remember, about a month ago Sam posted a comment along the lines of "AI will be capable of superhuman persuasion well before it is superhuman at general intelligence, which may lead to very strange outcomes".

Superhuman persuasion is Sam's area of expertise, so he would make that a priority when building chatbots.

Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

#552

Post by naasking »

cduzz wrote: post #316

I was talking to my (12 year old) son about parts of math he finds boring. He said that he thinks absolute value is absurdly easy and extremely boring. I asked him if there was anything that might make it more interesting, he said "maybe complex numbers". So I asked him "what would the absolute value of i+1 be?" he thinks for a little bit and says "square root of 2" and I ask him "what about the absolute value of 2i…

> this sort of understanding is a result of some manner of understanding the underlying mechanisms and not a result of just having a huge dictionary of synonyms.

He developed an understanding of the underlying mechanisms because he correlated concepts between algebraic and geometric domains, ie. multimodal training data. Multimodal models are already known to be meaningfully better than unimodal ones. We've barely scratched the surface of multimodal training.

Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

#553

Post by himaraya »

wbhart wrote: post #550
Davidzheng wrote: post #511

Earlier quoted context omitted.

I agree that in and of itself it's not enough to be alarmed. Also i have to say i don't really know what grade school mathematics means here(multiplication? Proving triangles are congruent?). But I think the question is, whether the breakthrough is an algorithmic change in reasoning. If it is, then it could challenge all 4 of your limitations. Again this article is low on details so really we are arguing over our bes…

I think maybe I didn't make myself quite clear here. There are already algorithms which can solve advanced mathematical problems 100% reliably (prove theorems). There are even algorithms which can prove any correct theorem that can be stated in a certain logical language, given enough time. There are even systems in which these algorithms have actually been implemented. My point is that no technology which can solve…

The article suggests the way Q* solves basic math problems matters more than the difficulty of the problems themselves. Either way, I think judging the claims made remains premature without seeing the supporting documentation.

Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

#554

Post by ugh123 »

gunapologist99 wrote: post #540
ugh123 wrote: post #530

Earlier quoted context omitted.

Are you saying an LLM can't come to the right conclusion and give an explanation for "what is the absolute value of 2i + 2"?

Are you saying it could , without having read it somewhere?

Maybe I'm unsure what we're arguing here. Did the guys kid drum that up himself or did he learn it from yt? Knowledge can be inferred or extracted. If it comes up with a correct answer and shows it's work, who cares how the knowledge was obtained?

Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

#555

Post by cduzz »

muskmusk wrote: post #503
cduzz wrote: post #316

I was talking to my (12 year old) son about parts of math he finds boring. He said that he thinks absolute value is absurdly easy and extremely boring. I asked him if there was anything that might make it more interesting, he said "maybe complex numbers". So I asked him "what would the absolute value of i+1 be?" he thinks for a little bit and says "square root of 2" and I ask him "what about the absolute value of 2i…

If you ask Ilya Sutskever he will say your kids head is full of neurons, so is LLMs. LLMs comsume training data and can then be asked questions. How different is that to your son watching YouTube and then answering questions? It's not 1:1 the same,yet, but it's in the neighborhood.

Well, my son is a meat robot who's constantly ingesting information from a variety of sources including but not limited to youtube. His firmware includes a sophisticated realtime operating system that models reality in a way that allows interaction with the world symbolically. I don't think his solving the |i+1| question was founded in linguistic similarity but instead in a physical model / visualization similarity.

So -- to a large degree "bucket of neurons == bucket of neurons" but the training data is different and the processing model isn't necessarily identical.

I'm not necessarily disagreeing as much as perhaps questioning the size of the neighborhood...

Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

#556

Post by richardw »

wbhart wrote: post #491

I feel very comfortable saying, as a mathematician, that the ability to solve grade school maths problems would not be at all a predictor of ability to solve real mathematical problems at a research level. The reason LLMs fail at solving mathematical problems is because: 1) they are terrible at arithmetic, 2) they are terrible at algebra, but most importantly, 3) they are terrible at complex reasoning (more specifica…

On backtracking, I thought tree-of-thought enabled that?

"considering multiple different reasoning paths and self-evaluating choices to decide the next course of action, as well as looking ahead or backtracking when necessary to make global choices"

https://arxiv.org/abs/2305.10601

Generally with you though, this thing is not leading to real smarts and that's accepted by many. Yes, it'll fill in a few gaps with exponentially more compute but it's more likely that an algo change is required once we've maxed out LLM's.

Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

#557

Post by nexuist »

_heimdall wrote: post #356

> OpenAI defines AGI as autonomous systems that surpass humans in most economically valuable tasks. Are we really defining intelligence as economic value at this point? This is completely ridiculous. We have yet to decide exactly what human intelligence, how it manifests in the body, or how to reliably measure it. I get that people want to justify developing artificial intelligence before understanding intelligence i…

If it makes money who cares if it counts as real intelligence? If we can automate ourselves into a post scarcity society then everyone and everything is free to be as intelligent and as stupid as it desires.

Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

#559

Post by himaraya »

caesil wrote: post #535
wbhart wrote: post #491

I feel very comfortable saying, as a mathematician, that the ability to solve grade school maths problems would not be at all a predictor of ability to solve real mathematical problems at a research level. The reason LLMs fail at solving mathematical problems is because: 1) they are terrible at arithmetic, 2) they are terrible at algebra, but most importantly, 3) they are terrible at complex reasoning (more specifica…

FWIW The Verge is reporting that people inside are also saying the Reuters story is bunk: https://www.theverge.com/2023/11/22/23973354/a-recent-openai...

> After being contacted by Reuters, OpenAI, which declined to comment, acknowledged in an internal message to staffers a project called Q* and a letter to the board before the weekend's events, one of the people said.

Reuters update 6:51 PST

The Verge has acted like an intermediary for Sam's camp during this whole saga, from my reading.

Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

#560

Post by xcv123 »

wbhart wrote: post #491

I feel very comfortable saying, as a mathematician, that the ability to solve grade school maths problems would not be at all a predictor of ability to solve real mathematical problems at a research level. The reason LLMs fail at solving mathematical problems is because: 1) they are terrible at arithmetic, 2) they are terrible at algebra, but most importantly, 3) they are terrible at complex reasoning (more specifica…

> I feel very comfortable saying, as a mathematician, that the ability to solve grade school maths problems would not be at all a predictor of ability to solve real mathematical problems at a research level.

At some point in the past, you yourself were only capable of solving grade school maths problems.

Post reply on HN