Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

reuters.com

Post reply on HN
Print view View original

921–930 of 1001 posts

Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

#921

Post by jug »

wbhart wrote: post #491

I feel very comfortable saying, as a mathematician, that the ability to solve grade school maths problems would not be at all a predictor of ability to solve real mathematical problems at a research level. The reason LLMs fail at solving mathematical problems is because: 1) they are terrible at arithmetic, 2) they are terrible at algebra, but most importantly, 3) they are terrible at complex reasoning (more specifica…

1. OpenAI researchers used loaded and emotional words, implying shock or surprise. It's not easy to impress an OpenAI researcher like this, and above all, they understand the difficulty difference between teaching AI grade school and complex math since many years. They also understand that solving math with any form of reliability is only an emergent property in quite advanced LLM's.

2. Often, research is made on toy models and if this would be such a model, acing grade school problems (as per the article) would be quite impressive to say the least as this ability simply isn't emergent early in current LLM's.

What I think might have happened here is a step forward in AI capacity that has surprised researchers not because it is able to do things it couldn't at all do before, but how _early_ it is able to do so.

Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

#922

Post by konschubert »

nopromisessir wrote: post #814
cduzz wrote: post #585

Earlier quoted context omitted.

Maybe; he still needs to finish his damned homework and remember to turn it in. And eat some vegetables.

All those things sound very boring to me. I can offer no concrete solutions. However, I have a friend who graduated from high school #1 of a big class and 2 years early. His mom explained that if he made at least a 1400(of 1600) on his SAT, she would buy him a new gaming computer. He then proceeded to make exactly a 1400. No more. No less. I recommend if you haven't tried already, an iteration to this approach using…

Be careful with reward systems, as it can destroy internal motivation.

Additionally, one very important thing to learn as a young adult is how to motivate yourself to do things that have only long term payoffs.

Of course I also understand that you can take SAT only once, so as bad as that is, it’s maybe not the best time to learn a life lesson.

Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

#923

Post by wegfawefgawefg »

jansan wrote: post #911
d--b wrote: post #777

Earlier quoted context omitted.

You make the asumption that Q* is a LLM, but I think OpenAI guys know very well that the current LLM architecture cannot achieve AGI. As the name suggests, this things is likely using some form of Q learning algorithm, which makes it closer to the DeepMind models than a transformer. My guess is that they pipe their LLM into some Q learnt net. The LLM may transform a natural language task into some internal representa…

There is a paper about something called Q*. I have no idea if they are connected or if the name matched coincidentially. https://arxiv.org/abs/2102.04518

The real world is a space of continuous actions. To this day Q algorithms have been ones of discrete action outputs. I'd be surprised if a Q algorithm could handle the huge action space of language. Honestly its weird they'd consider the Q family. I figured we were done with that after PPO performed so well.

Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

#924

Post by wegfawefgawefg »

d--b wrote: post #777
wbhart wrote: post #491

I feel very comfortable saying, as a mathematician, that the ability to solve grade school maths problems would not be at all a predictor of ability to solve real mathematical problems at a research level. The reason LLMs fail at solving mathematical problems is because: 1) they are terrible at arithmetic, 2) they are terrible at algebra, but most importantly, 3) they are terrible at complex reasoning (more specifica…

You make the asumption that Q* is a LLM, but I think OpenAI guys know very well that the current LLM architecture cannot achieve AGI. As the name suggests, this things is likely using some form of Q learning algorithm, which makes it closer to the DeepMind models than a transformer. My guess is that they pipe their LLM into some Q learnt net. The LLM may transform a natural language task into some internal representa…

As an ML programmer, i think that approach sounds really too complicated. It is always a bad idea to render the output of one neural network into output space before feeding it into another, rather than have them communicate in feature space.

Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

#925

Post by trashtester »

RamblingCTO wrote: post #789
sgt101 wrote: post #774

Earlier quoted context omitted.

Can LLM's compute any computable function? I thought that an LLM can approximate any computable function, if the function is within the distribution that it is are trained on. I think it's jolly interesting to think about different axiomizations in this context. Also we know that LLM's can't do a few things - arithmetic, inference & planning are in there. They look like they can because they retrieve discussions from…

I still believe that A(G)I will consist of subsystems and different network architectures (if NN's are the path to that), just like we humans have.

Many of the "specialist" parts of the brain are still made from cortical columns, though. Also, they are in many cases partly interchangeable, with some reduction in efficiency.

Transformers may be like that, in that they can do generalized learning from different types of input, with only minor modifications needed to optimize for different input (or output) modes.

Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

#926

Post by 93po »

HarHarVeryFunny wrote: post #896
Sol- wrote: post #832

Earlier quoted context omitted.

Thanks for the links, very interesting. Wonder how a "self-play" equivalent would look like for LLMs, since they have no easy criterion to evaluate how well they are doing like in Go (as mentioned in the videos).

I expect self-consistency might be one useful reward function. Of course in the real world, for a real intelligent system, reality is the feedback/reward system, but for an LLM limited to it's training set, with nothing to ground it, maybe this is the best you can do ... The idea is essentially that you need to assume (but of course GI-GO) that most of the training data is factual/reasonable whether in terms of facts…

Maybe I’m off mark here but it seems like video footage of real life would be a massively beneficial data set because it can watch these videos and predict what will happen one second into the future and then see if it was correct. And it can do this over millions of hours of footage and have billions of data points.

Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

#927

Post by 93po »

walthamstow wrote: post #847
Sol- wrote: post #832

Earlier quoted context omitted.

Thanks for the links, very interesting. Wonder how a "self-play" equivalent would look like for LLMs, since they have no easy criterion to evaluate how well they are doing like in Go (as mentioned in the videos).

ChatGPT does have some feedback that can be used to evaluate, in the form of thumbs up/down buttons, which probably nobody uses, and positive/negative responses to its messages. People often say "thanks" or "perfect!" in responses, including very smart people who frequent here.

They could also look at the use of the regenerate button, which I do use often, and would serve the same purpose

Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

#928

Post by jug »

JacobJeppesen wrote: post #772

Seems like they have made progress in combining reinforcement learning and LLMs. Andrej Karpathy mentions it in his new talk (~38 minutes in) [1], and Ilya Sutskever talks about it in a lecture at MIT (~29 minutes in) [2]. It would be a huge breakthrough to find a proper reward function to train LLMs in a reinforcement learning setup, and to train a model to solve math problems in a similar fashion to how AlphaGo use…

Q* may also be a reference to the well-known A* search algorithm but with this letter referring to Q-learning, further backing the reinforcement learning theory. https://en.wikipedia.org/wiki/Q-learning

Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

#929

Post by 93po »

jansan wrote: post #899
JacobJeppesen wrote: post #772

Seems like they have made progress in combining reinforcement learning and LLMs. Andrej Karpathy mentions it in his new talk (~38 minutes in) [1], and Ilya Sutskever talks about it in a lecture at MIT (~29 minutes in) [2]. It would be a huge breakthrough to find a proper reward function to train LLMs in a reinforcement learning setup, and to train a model to solve math problems in a similar fashion to how AlphaGo use…

Well, you could post a vast amount of comments into social media and see if and how others react to it. It's still humans doing the work, but they would not even know. If this was actually done (and this is just wild baseless speculation), this would be a good reason to let Sam go.

I see a lot of comments on reddit these days that are very clearly language models so it’s probably already happening on a large scale

Re: OpenAI researchers warned board of AI breakthrough ahead of CEO ouster

#930

Post by zeven7 »

J_cst wrote: post #652
cduzz wrote: post #316

I was talking to my (12 year old) son about parts of math he finds boring. He said that he thinks absolute value is absurdly easy and extremely boring. I asked him if there was anything that might make it more interesting, he said "maybe complex numbers". So I asked him "what would the absolute value of i+1 be?" he thinks for a little bit and says "square root of 2" and I ask him "what about the absolute value of 2i…

My son plays soccer

As someone who was thinking about the absolute value of complex numbers at that age, I wish I had played more soccer.
Post reply on HN