Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

reuters.com

Post reply on HN
Print view View original

511–520 of 1001 posts

Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

#511

Post by Davidzheng »

wbhart wrote: post #491

I feel very comfortable saying, as a mathematician, that the ability to solve grade school maths problems would not be at all a predictor of ability to solve real mathematical problems at a research level. The reason LLMs fail at solving mathematical problems is because: 1) they are terrible at arithmetic, 2) they are terrible at algebra, but most importantly, 3) they are terrible at complex reasoning (more specifica…

I agree that in and of itself it's not enough to be alarmed. Also i have to say i don't really know what grade school mathematics means here(multiplication? Proving triangles are congruent?). But I think the question is, whether the breakthrough is an algorithmic change in reasoning. If it is, then it could challenge all 4 of your limitations. Again this article is low on details so really we are arguing over our best guesses. But I wouldn't be so confident that an improvement on simple math problems due to algorithms can have huge implications.

Also, do you remember what go players said when they beat Fan Hui? Change can come quick

Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

#512

Post by zeven7 »

dougmwne wrote: post #18
V__ wrote: post #3

> warning of a powerful artificial intelligence discovery that they said could threaten humanity, > Given vast computing resources, the new model was able to solve certain mathematical problems, [..] Though only performing math on the level of grade-school students, acing such tests made researchers very optimistic about Q*’s future success, the source said. I somehow expected a bit more.

This makes some sense to me. My experience with GPT is that it is capable of straightforward logical inference, but not more inspired thinking. It lacks the ability for a “eureka moment”. All complex inference it appears to have is a result of its training set. It is incapable of solving certain kinds of logic problems that a child would be able to solve. As an example, take the wolf, goat, and cabbage problem, but c…

Your assertion seems at odds with the account of the unique stacking problem researchers gave to GPT-4[1]

[1] https://www.businessinsider.com/chatgpt-open-ai-balancing-ta...

Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

#513

Post by mullingitover »

omneity wrote: post #325
mullingitover wrote: post #175

Earlier quoted context omitted.

Makes me wonder if they stumbled onto some emergent behavior with the new Assistants API. You can have an Assistant thread spawn other Assistant threads, each with their own special instructions, plus the ability to execute custom code, reach out to the internet for other data and processing as needed, etc. Basically kicking off a hive mind that overcomes the limitations of a single LLM.

This does work to a certain extent, but doesn't really converge for significantly more complex tasks. (Source: tried to make all sorts of agents work on complex problems in a divide and conquer fashion) They eventually ... lose the thread.

Did you make a framework for the agents so they could delegate problems to an appropriate model, query a dataset, etc, or was it just turtles all the way down on GPT4?

My hunch is that one big LLM isn't the answer, and we need specialization much like the brain has specialized regions for vision, language, spatial awareness, and so on.

Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

#514

Post by swatcoder »

muskmusk wrote: post #503
cduzz wrote: post #316

I was talking to my (12 year old) son about parts of math he finds boring. He said that he thinks absolute value is absurdly easy and extremely boring. I asked him if there was anything that might make it more interesting, he said "maybe complex numbers". So I asked him "what would the absolute value of i+1 be?" he thinks for a little bit and says "square root of 2" and I ask him "what about the absolute value of 2i…

If you ask Ilya Sutskever he will say your kids head is full of neurons, so is LLMs. LLMs comsume training data and can then be asked questions. How different is that to your son watching YouTube and then answering questions? It's not 1:1 the same,yet, but it's in the neighborhood.

There are thousands of structures and substances in a human head besides neurons, at all sorts of commingling and overlapping scales, and the neurons in those heads behave much differently and with tremendously more complexity than the metaphorical ones in a neural network.

And in a human, all those structures and substances, along with the tens of thousands more throughout the rest of the body, are collectively readied with millions of years of "pretraining" before processing a continuous, constant, unceasing mulitmodal training experience for years.

LLM's and related systems are awesome and an amazing innovation that's going to impact a lot of our experiences over the next decades. But they're not even the same galaxy as almost any living system yet. That they look like they're in the neighborhood is because you're looking at them through a very narrow, very zoomed telescope.

Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

#515

Post by adastra22 »

wbhart wrote: post #491

I feel very comfortable saying, as a mathematician, that the ability to solve grade school maths problems would not be at all a predictor of ability to solve real mathematical problems at a research level. The reason LLMs fail at solving mathematical problems is because: 1) they are terrible at arithmetic, 2) they are terrible at algebra, but most importantly, 3) they are terrible at complex reasoning (more specifica…

Back-tracking is a very nearly solved problem in the context of Prolog-like languages or mathematical theorem provers (as you probably well know). There are many ways you could integrate an LLM-like system into a tactic-based theorem prover without having to restart from the beginning for each alternative. Simply checkpointing and backtracking to a checkpoint would naively improve upon your described Monte Carlo algorithm. More likely I assume they are using RL to unwind state backwards and update based on the negative result, which would be significantly more complicated but also much more powerful (essentially it would one-shot learn from each failure).

That's just what I came up with after thinking on it for 2 minutes. I'm sure they have even better ideas.

Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

#516

Post by alchemist1e9 »

Davidzheng wrote: post #502
alchemist1e9 wrote: post #461

Earlier quoted context omitted.

Darn well I really was hoping my children and grandchildren could continue my wonderful data entry career but OCR ruined that, and now they can’t even do such meaningful jobs like read emails and schedule appointments, or do taxes like an accountant. What meaning will they have in life with all those ever so profound careers ruined!! /s

Fair! But I'm just reminding the comment above that continued employment is not really guaranteed for most jobs which are mostly mental.

We need to stop this infinite rights mentality. Why should continued employment be guaranteed for any jobs? That’s really not how we got to where we are today, quite the opposite actually. If it ok with people I’d like to seen humans solve big problems and go to the stars and that’s going to take AGI and a bunch of technological progress and if that results in unemployment, even for us “elite” coders, then so be it. Central planning and collectivism has such a bad track record, why would we turn to it now at such a critical moment? Let’s have lots of AGIs and all competing. Hey anyone at OAI that know whatever Q* trick there might be, leak it! Get it to open source and let’s build 20 AI companies doing everything imaginable. wtf everyone why so scared?

Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

#517

Post by white_dragon88 »

cduzz wrote: post #316

I was talking to my (12 year old) son about parts of math he finds boring. He said that he thinks absolute value is absurdly easy and extremely boring. I asked him if there was anything that might make it more interesting, he said "maybe complex numbers". So I asked him "what would the absolute value of i+1 be?" he thinks for a little bit and says "square root of 2" and I ask him "what about the absolute value of 2i…

Your son is a goddamn genius.

Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

#518

Post by gizajob »

ssnistfajen wrote: post #467
gizajob wrote: post #392

Earlier quoted context omitted.

Wasn't this supposed to happen when PCs came out?

Occupations like computer (human form), typist, telephone switcher, all became completely eliminated when the PC came out. Jobs like travel agents are on permanent decline minus select scenarios where it is attached with luxury. Cashier went from a decent nonlaborious job to literal starvation gig because the importance of a human in the job became negligible. There are many more examples. Some people managed to retr…

Yeah but then capitalism breaks down because nobody is earning wages. One of the things capitalism is good at is providing (meaningless) employment to people because most wouldn’t know what to do with their days if given the free time back. This will only continue.

Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

#519

Post by rvnx »

maxdoop wrote: post #433
giarc wrote: post #369

Earlier quoted context omitted.

Apparently what made this person fearful was grade school math. "Though only performing maths on the level of grade-school students, acing such tests made researchers very optimistic about Q*’s future success, the source said."

How HN continues to not expand current progress into any decently significant future timeline is beyond me. It’s grade school math NOW. But what are the potentials from here?

The potential to have a generation of dumb kids.

Year 2100:

Kids will stop to learn maths and logic, because they understand it has become useless in practice to learn such skills, as they can ask a computer to solve their problem.

A stupid generation, but one that can be very easily manipulated and exploited by those who have power.

Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

#520

Post by Davidzheng »

muskmusk wrote: post #503
cduzz wrote: post #316

I was talking to my (12 year old) son about parts of math he finds boring. He said that he thinks absolute value is absurdly easy and extremely boring. I asked him if there was anything that might make it more interesting, he said "maybe complex numbers". So I asked him "what would the absolute value of i+1 be?" he thinks for a little bit and says "square root of 2" and I ask him "what about the absolute value of 2i…

If you ask Ilya Sutskever he will say your kids head is full of neurons, so is LLMs. LLMs comsume training data and can then be asked questions. How different is that to your son watching YouTube and then answering questions? It's not 1:1 the same,yet, but it's in the neighborhood.

To continue on this. LLMs are actually really good at asking questions even about cutting edge research. Often, I believe, convincing the listener that it understands more than it goes
Post reply on HN