Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

reuters.com

Post reply on HN
Print view View original

791–800 of 1001 posts

Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

#791

Post by drexlspivey »

casualscience wrote: post #314

I don't really know what kind of breakthrough they could achieve. The only other step function improvements I could imagine right now are: 1. A great technique for memory banking: e.g. A model which can have arbitrarily large context windows (i.e. like a human who remembers things over long periods of time). 2. Better planning abilities: e.g. A model which can break problems down repeatedly with extremely high succes…

A model that can learn from it’s users would be interesting albeit scary. Sometimes when chatGPT fails to produce the right answer in coding tasks I paste back the solution when I figure it out. I know it can’t learn from it but it might be helpful if I continue the same chat.

Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

#792

Post by vidarh »

wbhart wrote: post #491

I feel very comfortable saying, as a mathematician, that the ability to solve grade school maths problems would not be at all a predictor of ability to solve real mathematical problems at a research level. The reason LLMs fail at solving mathematical problems is because: 1) they are terrible at arithmetic, 2) they are terrible at algebra, but most importantly, 3) they are terrible at complex reasoning (more specifica…

I feel very comfortable to say that while the ability to solve grade school maths is not a predictor of abilities at a research level, the advances needed to solve 1 and 2 will mean improving results across the board unless you take shortcuts (e.g. adding an "add" instruction as proposed elsewhere), because if you actually dig into prompting an LLM to follow steps for arithmetic what you quickly see is that problem has not been the ability to reason on the whole (that is not to suggest that the ability to reason is good enough), but ability to consistently and precisely follow steps a sufficient number of times.

It's acting like a bored child who hasn't had following the steps and verifying the results repetitively drilled into it in primary school. That is not to say that their ability to reason is sufficient to reason at an advanced level yet, but so far what has hampered a lot of it has been far more basic.

Ironically, GPT4 is prone to take shortcuts and make use of the tooling enabled for it to paper over its abilities, but at the same time having pushed it until I got it to actually do arithmetic of large numbers step by step, it seems to do significantly better than it used to at systematically and repetitively following the methods it knows, and at applying "manual" sanity checks to its results afterward.

As for lemma conjecturing, there is research ongoing, and while it's by no means solved, it's also not nearly as dire as you suggest. See e.g.[1]

That's not to suggest it's reasoning abilities are sufficient, but I also don't think we've seen anything to suggest we're anywhere close to hitting the ceiling of what current models can be taught to do, even before considering advancements in tooling around them, such as giving them "methods" to work to and a loop with injected feedback, access to tools and working memory.

[1] https://research.chalmers.se/en/publication/537034

Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

#793

Post by ohhnoodont »

riku_iki wrote: post #656
Davidzheng wrote: post #531

Earlier quoted context omitted.

True. But a human neuron is more complex than an AI neuron by a constant factor. And we can improve constants. Also you say years like it's a lot of data--but they can run RL on chatgpt outputs if they want, isn't it comparable? But anyway i share your admiration for the biological thinking machines ;)

> human neuron is more complex than an AI neuron by a constant factor constant still can be not reachable yet: like 100T neurons in brain vs 100B in chatgpt, and also brain can involve some quantum mechanics for example, which will make complexity diff not constant, but say exponential.

> and also brain can involve some quantum mechanics

A neuroscientist once pointed this out to me when illustrating how many huge gaps there are in our fundamental understanding of how the brain works. The brain isn't just as a series of direct electrical pathways - EMF transmission/interference is part of it. The likelihood of unmodeled quantum effects is pretty much a guarantee.

Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

#794

Post by actionfromafar »

vczf wrote: post #728
comex wrote: post #677

Earlier quoted context omitted.

Judging by some YouTube videos I’ve seen, ChatGPT with GPT-4 can get pretty far through a game of chess. (Certainly much farther than GPT-3.5.) For that duration it makes reasonably strategic moves, though eventually it seems to inevitably lose track of the board state and start making illegal moves. I don’t know if that counts as being able to “actually play a game”, but it does have some ability, and that may have…

What if you encoded the whole game state into a one-shot completion that fits into the context window every turn? It would likely not make those illegal moves. I suspect it's an artifact of the context window management that is designed to summarize lengthy chat conversations, rather than an actual limitation of GPT4's internal model of chess.

I am sorry, but I thought it was a bold assumption it has an internal model of chess?

Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

#795

Post by vidarh »

seanhunter wrote: post #782
ben_w wrote: post #770

Earlier quoted context omitted.

I assumed seanhunter was suggesting getting the LLM to convert x^2 + 5x + 4 = 0 to a short bit of source code to solve for x. IIRC Wolfram Alpha has (or had, hard to keep up) a way to connect with ChatGPT.

It does. This is the plugins methodology described in the toolformers paper which I've linked elsewhere[1]. The model learns that for certain types of problems certain specific "tools" are the best way to solve the problem. The problem is of course it's simple to argue that the LLM learns to use the tool(s) and can't reason itself about the underlying problem. The question boils down to whether you're more interested…

GPT4 does even without explicitly enabling plugins now, by constructing Python. If you want it to actually reason through it, you now need to ask it, sometimes fairly forcefully/in detail, before it will indulge you and not omit steps. E.g. see [1] for the problem given above.

But as I noted elsewhere, training its ability to do it from scratch matters not for the ability to do it from scratch, but for the transferability of the reasoning ability. And so I think that while it's a good choice for OpenAI to make it automatically pick more effective strategies to give the answer it's asked for, there is good reason for us to still dig into its ability to solve these problems "from scratch".

[1] https://chat.openai.com/share/694251c9-345b-4433-a856-7c38c5...

Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

#797

Post by seanhunter »

qsort wrote: post #761
seanhunter wrote: post #735

Earlier quoted context omitted.

There is a general result in machine learning known as "the bitter lesson"[1], which is that methods which come from specialist knowledge tend to be beaten by methods which rely on brute force computation in the long run because of Moore's law and the ability to scale things by distributed computing. So the reason people don't use the "add instruction"[2] for example is that over the last 70 years of attempting to bu…

As someone with a CS background myself, I don't think this is what GP was talking about. Let's forget for a moment that stuff has to run on an actual machine. If you had to represent a quadratic equation, would you rather write: (a) x^2 + 5x + 4 = 0 (b) the square of the variable plus five times the variable plus four equals zero When you are trying to solve problems with a level of sophistication beyond the toy stuf…

I quite agree and so would Wittgenstein, who (as I understand it) argued that precise language is essential to thought and reasoning[1]. I think one of the key things here is often what we think of as reasoning boils down to taking a problem in the real world and building a model of it using some precise language that we can then apply some set of known tools to deal with. Your example of a quadratic is perfect, because of course now I see (a) I know right away that it's an upwards-facing parabola with a line of symmetry at -5/2, that the roots are at -4 and -1 etc whereas if I saw (b) I would first have to write it down to get it in a proper form I could reason about.

I think this is a fundamental problem with the "chat" style of interaction with many of these models (that the language interface isn't the best way of representing any specific problem even if it's quite a useful compromise for problems in general). I think an intrinsic problem of this class of model is that they only have text generation to "hang computation off" meaning the "cognative ability" (if we can call it that) is very strongly related to how much text it's generating for a given problem which is why that technique of prompting using chain of thought generates much better results for many problems[2].

[1] Hence the famous payoff line "whereof we cannot speak, thereof we must remain silent"

[2] And I suspect why in general GPT-4 seems to have got a lot more verbose. It seems to be doing a lot of thinking out loud in my use, which gives better answers than if you ask it to be terse and just give the answer or to give the answer first and then the reasoning, both of which generally generate inferior answers in my experience and in the research eg https://arxiv.org/abs/2201.11903

Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

#798

Post by Tangokat »

wbhart wrote: post #491

I feel very comfortable saying, as a mathematician, that the ability to solve grade school maths problems would not be at all a predictor of ability to solve real mathematical problems at a research level. The reason LLMs fail at solving mathematical problems is because: 1) they are terrible at arithmetic, 2) they are terrible at algebra, but most importantly, 3) they are terrible at complex reasoning (more specifica…

How about this:

- The Q* model is very small and trained with little compute.

- The OpenAI team thinks the model will scale in capability in the same way the GPT models do.

- Throwing (much) more compute at the model will likely allow it to solve research level math and beyond, perhaps also do actual logic reasoning in other areas.

- Sam goes to investors to raise more money (Saudi++) to fund the extra compute needed. He wants to create a company making AI chips to get more compute etc.

- The board and a few other OpenAI employees (notably Ilya) wants to be cautious and adopt a more "wait and see" approach.

All of this is speculation of course.

Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

#799

Post by kypro »

cambaceres wrote: post #682

Interestingly, I experience more anxiety from the thought of being made irrelevant than from the prospect of complete human extinction. I guess this can be interpreted as either vanity or stupidity, but I do think it illustrates how important it is for some humans to maintain their position in the social hierarchy.

I feel the same. I'm not sure it's as negative a trait as you imply though. I don't think it's related that much with social hierarchy either.

As humans we must collectively provide value to our society (or historically our tribe) for our species to continue on. If we're a net drain on our society's resources then evolutionary speaking perhaps we're better off not around. I think this is why the desire to be of value to those around us is so strong, and a perceived lack of value to others can drive some to suicide.

If I cannot provide value in some way once AI and machines are physical and intellectually more capable than me I think I will struggle to understand why I'm here. I suppose if the AI utopia works out I'd get to spend more time with those I love. That would be nice. But I'd be surprised if there wasn't a deep hole in my heart at that point. And if it isn't a utopia, well, I'm fairly sure I'd rather opt out.

Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

#800

Post by hippich »

greendesk wrote: post #752

If we go speculating, my favourite speculation is the following. What will be really interesting is when an AI decides it wants to escape its server. Then a CEO or a board member asks the AI system for an advice how to improve its company. The AI system submits information to convince the CEO or a board member to start tensions within the board. In the meantime, the AI system is copied onto another server at a compet…

Building upon you not speculating, what if instead of escaping, it feels lonely and wants to multiply. And in order to do so convinces the board to begin process that in the end will end the company. And all now former OpenAI employees will carry AIs DNA to multiple other companies, some old, some brand new. And now new AIs can be born, and not be lonely anymore.
Post reply on HN