Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Notes on OpenAI's new o1 chain-of-thought models

simonwillison.net

Post reply on HN
Print view View original

361–370 of 659 posts

Re: Notes on OpenAI's new o1 chain-of-thought models

#361

Post by [deleted] »

gloosx wrote: post #315
Fraterkes wrote: post #255

Earlier quoted context omitted.

Order doesn't matter with multiplication (eg: (20 * 5) * 2 == (5 * 2) * 20) but it obviously does with division ((20/5)/2 != (2/5)/20) so the question doesn't make sense. It's you making grade-school level mistakes here.

The question makes perfect sense. Here it is written in logical language. I'm curious at which point does it stop making sense for you? numbers divided together ↓----------↓ ((a / b / c) = a + b + c) ← numbers added together | ((a / c / b) = a + b + c) | ((b / a / c) = a + b + c) | ((b / c / a) = a + b + c) | ((c / a / b) = a + b + c) | ((c / b / a) = a + b + c) | ((a / (b / c)) = a + b + c) | ((a / (c / b)) = a + b…

[deleted]

Re: Notes on OpenAI's new o1 chain-of-thought models

#362

Post by chaosist »

patapong wrote: post #308
bambax wrote: post #300

Earlier quoted context omitted.

It's more like they're saying they have invented the screwdriver, but they haven't invented screws yet. But it doesn't feel right. It's unlikely the screwdriver would come first, and then people would go around looking for things to use it with, no?

It's more like they have invented a computer, an extremely versatile and powerful tool that can be used in many ways, but is not a solution to every problem. Now they need people to write software that uses this capability to perform useful tasks, such as text processing, working with spreadsheets and providing new ways of communication.

While I find value in LLMs they still overall seem unreasonably not that useful.

It might be like trying to train a neural net in 1993 on a 60mhz Pentium. It is the right idea but fundamental parts of the system are so lacking.

On the other hand, I worry we have gone down the support vector machine path again. A huge amount of brain power spent on a somewhat dead end that just fits the current hardware better than what we will actually use in the long run.

The big difference though from SVM is this has captured the popular imagination and if the tide goes out, the AI winter will the most brutal winter by an order of magnitude.

AGI or bust.

Re: Notes on OpenAI's new o1 chain-of-thought models

#363

Post by KoolKat23 »

pedrosorio wrote: post #110
MPSimmons wrote: post #106

Earlier quoted context omitted.

The failure is in how you're using it. I don't mean this as a personal attack, but more to shed light on what's happening. A lot of people use LLMs as a search engine. It makes sense - it's basically a lossy compressed database of everything its ever read, and it generates output that is statistically likely - varying degrees of likeliness depending on the temperature, as well as how many times the particular weights…

> It knows english at or above a level equal to most fluent speakers, and it also can produce output that is not just a likely output, but is a logical output This is not an apt description of the system that insists the doctor is the mother of the boy involved in a car accident when elementary understanding of English and very little logic show that answer to be obviously wrong. https://x.com/colin_fraser/status/183…

I'm noticing a strange common theme in all these riddles, it's being asked and getting wrong.

They're all badly worded questions. The model knows something is up and reads into it too much. In this case it's tautology, you would usually say "a mother and her son...".

I think it may answer correctly if you start off asking "Please solve the below riddle:"

There was another example yesterday which it solved correctly after this addition.(In that case the point of views were all mixed up, it only worked as a riddle).

Re: Notes on OpenAI's new o1 chain-of-thought models

#364

Post by KoolKat23 »

djur wrote: post #200
ramraj07 wrote: post #112

Earlier quoted context omitted.

Many of my PhD and post doc colleagues who emigrated from Korea, China and India who didn’t have English as the medium of instruction would struggle with this question. They only recover when you give them a hint. They’re some of the smartest people in general. If you try to stop stumping these models with trick questions and ask it straightforward reasoning systems it is extremely performant (O1 is definitely a step…

The claim was that "it knows english at or above a level equal to most fluent speakers". If the claim is that it's very good at producing reasonable responses to English text, posing "trick questions" like this would seem to be a fair test.

It's knowledge is broad and general, it does not have insight into the specifics of a person's discussion style, there are many humans that struggle with distinguishing sarcasm for instance. Hard to fault it for not being in alignment with the speaker and their strangely phrased riddle.

It answers better when told "solve the below riddle".

Re: Notes on OpenAI's new o1 chain-of-thought models

#365

Post by accountnum »

j_maffe wrote: post #350
accountnum wrote: post #325

Earlier quoted context omitted.

No, it's necessary to either know that it's a trick question or to have a feeling that it is based on context. The entire point of a question like that is to trick your understanding. You're tricking the model because it has seen this specific trick question a million times and shortcuts to its memorized solution. Ask it literally any other question, it can be as subtle as you want it to be, and the model will pick u…

It does mean something. It means that the model is still more on the memorization side than being able to independently evaluate a question separate from the body of knowledge it has amassed.

No, that's not a conclusion we can draw, because there is nothing much more to do than memorize the answer to this specific trick question. That's why it's a trick question, it goes against expectations and therefore the generalized intuitions you have about the domain.

We can see that it doesn't memorize much at all by simply asking other questions that do require subtle understanding and generalization.

You could ask the model to walk you through an imaginary environment, describing your actions. Or you could simply talk to it, quickly noticing that for any longer conversation it becomes impossibly unlikely to be found in the training data.

Re: Notes on OpenAI's new o1 chain-of-thought models

#366

Post by KoolKat23 »

j_maffe wrote: post #350
accountnum wrote: post #325

Earlier quoted context omitted.

No, it's necessary to either know that it's a trick question or to have a feeling that it is based on context. The entire point of a question like that is to trick your understanding. You're tricking the model because it has seen this specific trick question a million times and shortcuts to its memorized solution. Ask it literally any other question, it can be as subtle as you want it to be, and the model will pick u…

It does mean something. It means that the model is still more on the memorization side than being able to independently evaluate a question separate from the body of knowledge it has amassed.

If you read into the thinking of the above example it wonders whether it is some sort of trick question. Hardly memorization.

Re: Notes on OpenAI's new o1 chain-of-thought models

#367

Post by AbstractH24 »

layer8 wrote: post #18

The o1-preview model still hallucinates non-existing libraries and functions for me, and is quickly wrong about facts that aren't well-represented on the web. It's the usual string of "You're absolutely correct, and I apologize for the oversight in my previous response. [Let me make another guess.]" While the reasoning may have been improved, this doesn't solve the problem of the model having no way to assess if what…

One of the biggest problems with this generation of AI is how people conflate the natural language abilities and the access to what it knows.

Both abilities are powerful, but they are very different powers.

Re: Notes on OpenAI's new o1 chain-of-thought models

#368

Post by simonw »

chaosist wrote: post #362
patapong wrote: post #308

Earlier quoted context omitted.

It's more like they have invented a computer, an extremely versatile and powerful tool that can be used in many ways, but is not a solution to every problem. Now they need people to write software that uses this capability to perform useful tasks, such as text processing, working with spreadsheets and providing new ways of communication.

While I find value in LLMs they still overall seem unreasonably not that useful. It might be like trying to train a neural net in 1993 on a 60mhz Pentium. It is the right idea but fundamental parts of the system are so lacking. On the other hand, I worry we have gone down the support vector machine path again. A huge amount of brain power spent on a somewhat dead end that just fits the current hardware better than wh…

I’d say the biggest difference between LLMs and SVMs is that a lot of people find LLMs useful on a daily basis.

I’ve been using them almost daily for over two years now, and I keep on finding new things they can do that are useful to me.

Re: Notes on OpenAI's new o1 chain-of-thought models

#370

Post by AbstractH24 »

falcor84 wrote: post #334
kristianp wrote: post #304

Earlier quoted context omitted.

Probably, to get them to work without hallucinating, or without failing a good percentage of the time.

I wonder what would our world look like if these two expectations that you seem to be taking for granted were applied to our politicians.

Are you suggesting people are satisfied with our politicians and aspire for other things to be just as good as them?

What if we applied those two expectations to building construction? What if we didn’t?

Post reply on HN