Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Notes on OpenAI's new o1 chain-of-thought models

simonwillison.net

Post reply on HN
Print view View original

541–550 of 659 posts

Re: Notes on OpenAI's new o1 chain-of-thought models

#541

Post by richerram »

williamdclt wrote: post #164
MPSimmons wrote: post #106

Earlier quoted context omitted.

The failure is in how you're using it. I don't mean this as a personal attack, but more to shed light on what's happening. A lot of people use LLMs as a search engine. It makes sense - it's basically a lossy compressed database of everything its ever read, and it generates output that is statistically likely - varying degrees of likeliness depending on the temperature, as well as how many times the particular weights…

> Treat it as a naive but intelligent intern That’s the problem: it’s a _terrible_ intern. A good intern will ask clarifying questions, tell me “I don’t know” or “I’m not sure I did it right”. LLMs do none of that, they will take whatever you ask and give a reasonable-sounding output that might be anything between brilliant and nonsense. With an intern, I don’t need to measure how good my prompting is, we’ll usually…

I think this is the main issue with these tools... what people are expecting of them.

We have swallowed the pill that LLMs are supposed to be AGI and all that mumbo jumbo, when they are just great tools and as such one needs to learn to use the tool the way it works and make the best of it, nobody is trying to hammer a nail with a broom and blaming the broom for not being a hammer...

Re: Notes on OpenAI's new o1 chain-of-thought models

#542

Post by mft_ »

causal wrote: post #536
aubanel wrote: post #296

Earlier quoted context omitted.

He's speaking about his objective to make ever stronger LLMs: so for this his secondary objective is to measure their real performance. The human preference is not that good of a proxy measurement: for instance, it can be gamed by making the model more assertive, causing the human error-spotting ability to decrease a lot [0]. So what he's really saying is that non-rigorous human vibe checks (like those LMSys Chatbot…

It's trivial to come up with prompts that 4o fails. If it's hard to come up with prompts that 1o succeeds on but 4o fails, that implies the delta is not that great.

Or, the delta depends on the nature of the problem/prompt, we’ve not yet figured that out, there’s a relatively narrow range of prompts with large delta, and so finding those examples is a work in progress?

Re: Notes on OpenAI's new o1 chain-of-thought models

#543

Post by rm_-rf_slash »

causal wrote: post #537
briandw wrote: post #509

My 12 YO and I just built a fishing game using o1 preview. Prompt: "make a top down game in pyxel. the play has to pay off a debt to a cat by catching fish. the goal is for the player to catch the giant king fish. To catch the king fish the player needs to sell the fish to the cat and get money to buy better rods, 3 levels of rod, last one can catch the king fish." It nailed the execution. Amazing.

My first few attempts at getting it to work with an existing codebase have not been impressive. Perhaps 1o is best suited to difficult problems that can be stated in only a few sentences.

I’ve had the opposite experience: terrific use with modifying existing codebases. But then again I’ve been using GPT4 to code for over a year now and so I’m used to writing out prompts with my eyes closed.

Re: Notes on OpenAI's new o1 chain-of-thought models

#544

Post by bsenftner »

MPSimmons wrote: post #106
layer8 wrote: post #18

The o1-preview model still hallucinates non-existing libraries and functions for me, and is quickly wrong about facts that aren't well-represented on the web. It's the usual string of "You're absolutely correct, and I apologize for the oversight in my previous response. [Let me make another guess.]" While the reasoning may have been improved, this doesn't solve the problem of the model having no way to assess if what…

The failure is in how you're using it. I don't mean this as a personal attack, but more to shed light on what's happening. A lot of people use LLMs as a search engine. It makes sense - it's basically a lossy compressed database of everything its ever read, and it generates output that is statistically likely - varying degrees of likeliness depending on the temperature, as well as how many times the particular weights…

> Treat it as a naive but intelligent intern.

I've found an amazing amount of success with a three step prompting method that appears to create incredibly deep subject matter experts who then collaborate with the user directly.

1) Tell the LLM that it is a method actor, 2) Tell the method actor they are playing the role of a subject matter expert, 3) At each step, 1 and 2, use the technical language of that type of expert; method actors have their own technical terminology, use it when describing the characteristics of the method actor, and likewise use the scientific/programming/whatever technical jargon of the subject matter expert your method actor is playing.

Then, in the system prompt or whatever logical wrapper the LLM operates through for the user, instruct the "method actor" like you are the film director trying to get your subject matter expert performance out of them.

I offer this because I've found it works very well. It's all about crafting the context in which the LLM operates, and this appears to cause the subject matter expert to be deeper, more useful, smarter.

Re: Notes on OpenAI's new o1 chain-of-thought models

#545

Post by MPSimmons »

EagnaIonat wrote: post #376
MPSimmons wrote: post #106

Earlier quoted context omitted.

The failure is in how you're using it. I don't mean this as a personal attack, but more to shed light on what's happening. A lot of people use LLMs as a search engine. It makes sense - it's basically a lossy compressed database of everything its ever read, and it generates output that is statistically likely - varying degrees of likeliness depending on the temperature, as well as how many times the particular weights…

> Treat it as a naive but intelligent intern. You are falling into the trap that everyone does. In anthropomorphising it. It doesn't understand anything you say. It just statistically knows what a likely response would be. Treat it as text completion and you can get more accurate answers.

Oh no, I'm well aware that it's a big file full of numbers. But when you chat with it, you interact with it as though it were a person so you are necessarily anthropomorphizing it, and so you get to pick the style of the interaction.

(In truth, I actually treat it in my mind like it's the Enterprise computer and I'm Beverly Crusher in "Remember Me")

Re: Notes on OpenAI's new o1 chain-of-thought models

#546

Post by B1FF_PSUVM »

AFAICT, we got the ELIZA 60th anniversary edition, and are now headed for some Prolog/production systems iteration.

One of these days those contraptions will work well enough, not because they're perfect, but because human intelligence isn't really that good either.

(And looking in this mirror isn't flattering us any.)

Re: Notes on OpenAI's new o1 chain-of-thought models

#547

Post by bgun »

a_wild_dandan wrote: post #510
bgun wrote: post #484

Earlier quoted context omitted.

I agree. My example for something “AI” should be able to do is to create a CAD model for the Empire State Building or the Parthenon based on known facts and photos. I don’t think these are “moving the goalposts” examples, they are things that an actual intelligence capable of passing a PhD physics exam should be able to do.

I mean, I passed a physics PhD exam and I can’t model the Empire State Building. The jury is still out on whether I’m an intelligence tho.

My point is that you could, given enough time and all the information available to you online about these well-documented buildings. You could learn CAD and figure out a reasonable way to output a 3D model, because you can think and reason spatially. The current batch of AI tools can regurgitate complex facts, but they can't actually think in 3D like an being that spends its life navigating physical spaces.

Maybe I'm wrong and we are well on our way to AI tools for this, but right now if I tell any of the current generation of image models to do something like "rotate object 70 degrees, tilt camera down 20 degrees and re-render" then what comes out is never even approximately close.

Re: Notes on OpenAI's new o1 chain-of-thought models

#548

Post by freejazz »

awb wrote: post #465
berniedurfee wrote: post #381

Earlier quoted context omitted.

I’m starting to think this is an unsolvable problem with LLMs. The very act of “reasoning” requires one to know that they don’t know something. LLMs are giant word Plinko machines. A million monkeys on a million typewriters. LLMs are not interns. LLMs are assumption machines. None of the million monkeys or the collective million monkeys are “reasoning” or are capable of knowing. LLMs are a neat parlor trick and are s…

It probably depends on your problem space. In creative writing, I wonder if its even perceptible if the LLM is creating content at the boundaries of its knowledge base. But for programming or other falsifiable (and rapidly changing) disciplines it is noticeable and a problem. Maybe some evaluation of the sample size would be helpful? If the LLM has less than X samples of an input word or phrase it could include a cau…

> I wonder if its even perceptible if the LLM is creating content at the boundaries of its knowledge base

The problem space in creative writing is well beyond the problem space for programming or other "falsifiable disciplines".

Re: Notes on OpenAI's new o1 chain-of-thought models

#549

Post by AbstractH24 »

falcor84 wrote: post #433
AbstractH24 wrote: post #370

Earlier quoted context omitted.

Are you suggesting people are satisfied with our politicians and aspire for other things to be just as good as them? What if we applied those two expectations to building construction? What if we didn’t?

I think it's always good to aspire for more, but we shouldn't be expecting perfect results in novel areas of technology. Taking up your construction metaphor, LLMs are now where construction was perhaps 3000 years ago; buildings weren't that sturdy, but even if the roofs leaked a bit, I'm sure it beat sleeping outside on a rainy night. We need to continue iterating.

Continuing this metaphor further, 3000 years ago built a tower to the sky called the Tower of Babel.

Re: Notes on OpenAI's new o1 chain-of-thought models

#550

Post by happypumpkin »

zurfer wrote: post #226

> I asked on Twitter for examples of prompts that people had found which failed on GPT-4o but worked on o1-preview. it seems trivial, but I tried for more than 2 hours in the past to get gpt4 to play tic-tac-toe optimally and failed (CoT prompt,etc.). The result were too many illegal moves and absolutely no optimal strategy. o1-preview can do it really well [1] However, when I use a non-standard grid (3x5) it fails t…

The new model does play very well but when it draws the board it frequently places the moves in incorrect locations (but seemingly still keeps track of the correct ones). But I can't fault it too much, I don't think what is essentially ASCII art is intended to be a strength of the model.

Edit: Actually third game with it led to it making an illegal move, and claiming a draw (which would've been inevitable given optimal play for the rest of the game but there were several valid moves left to make).

Post reply on HN