Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Notes on OpenAI's new o1 chain-of-thought models

simonwillison.net

Post reply on HN
Print view View original

631–640 of 659 posts

Re: Notes on OpenAI's new o1 chain-of-thought models

#631

Post by Mattclosson »

uptownfunk wrote: post #493
bambax wrote: post #281

Near the end, the quote from OpenAI researcher Jason Wei seems damning to me: > Results on AIME and GPQA are really strong, but that doesn’t necessarily translate to something that a user can feel. Even as someone working in science, it’s not easy to find the slice of prompts where GPT-4o fails, o1 does well, and I can grade the answer. But when you do find such prompts, o1 feels totally magical. We all need to find…

This hits the nail on the head. It is a consumer facing product not a technology to solve deep thinking.

i don't think that's what he's saying

Re: Notes on OpenAI's new o1 chain-of-thought models

#632

Post by kranuck »

> For example, in the future we may wish to monitor the chain of thought for signs of manipulating the user

I'm sick of these clowns couching everything in "look how amazing and powerful and dangerous out AI is"

This is in their excuse for why they hid a bunch of model output they still charge you for.

Re: Notes on OpenAI's new o1 chain-of-thought models

#633

Post by kranuck »

layer8 wrote: post #18

The o1-preview model still hallucinates non-existing libraries and functions for me, and is quickly wrong about facts that aren't well-represented on the web. It's the usual string of "You're absolutely correct, and I apologize for the oversight in my previous response. [Let me make another guess.]" While the reasoning may have been improved, this doesn't solve the problem of the model having no way to assess if what…

That problems feels somewhat fundamental to saying that these things have any ability to reason at all.

Re: Notes on OpenAI's new o1 chain-of-thought models

#634

Post by kranuck »

williamdclt wrote: post #164
MPSimmons wrote: post #106

Earlier quoted context omitted.

The failure is in how you're using it. I don't mean this as a personal attack, but more to shed light on what's happening. A lot of people use LLMs as a search engine. It makes sense - it's basically a lossy compressed database of everything its ever read, and it generates output that is statistically likely - varying degrees of likeliness depending on the temperature, as well as how many times the particular weights…

> Treat it as a naive but intelligent intern That’s the problem: it’s a _terrible_ intern. A good intern will ask clarifying questions, tell me “I don’t know” or “I’m not sure I did it right”. LLMs do none of that, they will take whatever you ask and give a reasonable-sounding output that might be anything between brilliant and nonsense. With an intern, I don’t need to measure how good my prompting is, we’ll usually…

> have no idea whether the LLM understood what I’m asking

That's easy. The answer is it doesn't. It has no understanding of anything it does.

> if it’s able to do it

This is the hard part.

Re: Notes on OpenAI's new o1 chain-of-thought models

#635

Post by TeMPOraL »

grey-area wrote: post #615
TeMPOraL wrote: post #469

Earlier quoted context omitted.

> So, as usual, ChatGPT fails to answer the modified riddle and gives the plagiarized stock answer and explanation to the original one. No intelligence here. Or, fails in the same way any human would, when giving a snap answer to a riddle told to them on the fly - typically, a person would recognize a familiar riddle half of the first sentence in, and stop listening carefully, not expecting the other party to give th…

I'm curious what you think is happening here as your answer seems to imply it is thinking (and indeed rushing to an answer somehow). Do you think the generative AI has agency or a thought process? It doesn't seem to have anything approaching that to me, nor does it answer quickly. It seems to be more like a weighing machine based on past tokens encountered together, so this is exactly the kind of answer we'd expect o…

My thinking is that LLMs are very similar, perhaps structurally the same, as a piece of human brain that does the "inner voice" thing. The boundary between the subconscious and conscious, that generates words and phrases and narratives pretty much like "feels best" autocomplete[0] - bits that other parts of your mind evaluate and discard, or circle back, because if you were just to say or type directly what your inner voice says, you'd sound like... a bad LLM.

In my own experience, when I'm asked a question, my inner voice starts giving answers immediately, following associations and what "feels right"; the result is eerily similar to LLMs, particularly when they're hallucinating. The difference is, you see the immediate output of an LLM; with a person, you see/hear what they choose to communicate after doing some mental back-and-forth.

So I'm not saying LLMs are thinking - mostly for the trivial reason of them being exposed through low-level API, without built-in internal feedback loop. But I am saying they're performing the same kind of thing my inner voice does, and at least in my case, my inner voice does 90% of my "thinking" day-to-day.

--

[0] - In fact, many years before LLMs were a thing, I independently started describing my inner narrative as a glorified Markov chain, and later discovered it's not an uncommon thing.

Re: Notes on OpenAI's new o1 chain-of-thought models

#636

Post by gabrielrdz »

franze wrote: post #402

Just coded this this morning using chatgpt o1 - it is the reimplementation of an old idea now music, multiple dots, more and more bug fixes honestly, chatgpt is now a better coder than i ever was or will be https://lsd.franzai.com/

hah, this takes me back. There used to be a game called Jezzball, I think, back in the late 90's or early 00's. Had a lot of fun with that one.

Re: Notes on OpenAI's new o1 chain-of-thought models

#637

Post by dartos »

KoolKat23 wrote: post #611
dartos wrote: post #601

Earlier quoted context omitted.

At a fundamental level, brains don’t operate on floating point numbers encoded in bits. They have chemicals to facilitate electrochemical reactions which can affect how they respond to input. They don’t throw away all knowledge of what they just said. They change continuously, not just in fixed training loops. They don’t operate in turns. I could go on. Honestly the number of people who just heard “learning,” “neural…

Fundamentally and physically are two different things. A logic gate is a logic gate if it's in neurons or silicon. Are abacus and calculators solving different things? No. You're proving my point, things like them changing continuously are exactly what I mean when I say the brain is more efficient. Where there's a will theres a way and our brains are evidence that it can be done.

You're saying that because two different objects can to solve the same problem, they must work the same way.

An abacus and a calculator were both made to solve relativly simple math problems, so they must work in the same way, right?

And apple and an orange are ways to store sugar for plants, so they must be the same thing, right?

No. That's not how any of this works. An abacus and a calculator are two different tools that solve the same problem. They don’t act like each other just because the abstract outcome is the same

> You're proving my point, things like them changing continuously are exactly what I mean when I say the brain is more efficient.

I don't see how that proves that neural networks act like brains.

It's also not just a difference in terms of efficiency, it's the fundamental way that statistical models like neural networks are trained. Every time their trained, it's a brand new model, unlike a brain, which is still the same brain.

Also, neural networks and brains were NOT made to solve the same problems... even if your argument made any sense, it doesn't fit here.

Re: Notes on OpenAI's new o1 chain-of-thought models

#638

Post by nbzso »

jes5199 wrote: post #483
nbzso wrote: post #457

Working in tech for over 30 years. This is the first time when I don't see proposed technology as a valuable tool. Especially LLM's. Vastly overhyped, driven by pure greed and speculative narratives, limited implementation and high energy cost. Non-transparent. Errors marketed as a hallucination.

this is exactly what i said about the iphone

Sorry, there is no parallel between technology with direct implication and dreams from VC's and investors with low level of tech literacy.

Re: Notes on OpenAI's new o1 chain-of-thought models

#639

Post by KoolKat23 »

dartos wrote: post #637
KoolKat23 wrote: post #611

Earlier quoted context omitted.

Fundamentally and physically are two different things. A logic gate is a logic gate if it's in neurons or silicon. Are abacus and calculators solving different things? No. You're proving my point, things like them changing continuously are exactly what I mean when I say the brain is more efficient. Where there's a will theres a way and our brains are evidence that it can be done.

You're saying that because two different objects can to solve the same problem, they must work the same way. An abacus and a calculator were both made to solve relativly simple math problems, so they must work in the same way, right? And apple and an orange are ways to store sugar for plants, so they must be the same thing, right? No. That's not how any of this works. An abacus and a calculator are two different tool…

No I'm not saying they must work the same way. I'm saying it's evidence there is a more efficient way as they both solve the same problem and one is more efficient (in truth both are more efficient in different areas). At an abstract level they can be doing the same thing. What does a simulator do?

Think a little further yes, currently it's a brand new model each time but why will it be this way forever? Its an engineering problem one that we can solve and the brain is evidence it can be done.

Neural networks were originally inspired by the brain. Yes, they've deviated but there's absolutely no reason they can't take further inspiration.

Re: Notes on OpenAI's new o1 chain-of-thought models

#640

Post by KoolKat23 »

troupo wrote: post #629
KoolKat23 wrote: post #613

Earlier quoted context omitted.

Why does it need to answer more than that? You underestimate the opportunity that exists for automation out there. In my own case I've used it to make simple custom browser extensions transcribing PDFs, I don't have the time and wouldn't of made the effort to make the extension myself, the task would of continued to be done manually. It took two hours to make and it works, that's all I need in this case. Perfection i…

> Perfection is the enemy of good. Where exactly did I write anything about perfection? For me "AIs" are incapable of producing working code: https://news.ycombinator.com/item?id=41534233

You said you have to babysit each line of code, I mean this is simply untrue, if it works there's no need to babysit, the only reason you'd need to babysit every single line is if you're looking for perfection or it's something very obscure or unheard of.

Your example is perhaps valid, but there are other examples where it does work as I mentioned. I think it may be imprecise prompting, too general or with too little logic structure. It's not like Google search, the more detail and more technical you speak the better, assume it's a very precise expert. Its intelligence is very general so it needs precision to avoid confusing subject matter. A well structured logic to your request also helps as it's reasoning isn't the greatest.

Good prompting and verifying output is often still faster than manually typing it all.

Post reply on HN