Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Notes on OpenAI's new o1 chain-of-thought models

simonwillison.net

Post reply on HN
Print view View original

441–450 of 659 posts

Re: Notes on OpenAI's new o1 chain-of-thought models

#441

Post by rainsford »

ryanjshaw wrote: post #259
lucubratory wrote: post #182

Earlier quoted context omitted.

Yeah, I think what a lot of people miss about these sort of gotchas are that most of them were invented explicitly to gotcha humans, who regularly get got by them. This is not a failure mode unique to LLMs.

Yes it's so strange seeing people who clearly know these are 'just' statistical language models pat themselves on the back when they find limits on the reasoning capabilities - capabilities which the rest of us are pleasantly surprised exist to the extent they do in a statistical model, and happy to have access to for $20/mo.

It's because at least some portion of "the rest of us" talk as if LLMs are far more capable than they really are and AGI is right around the corner, if not here already. I think the gotchas that play on how LLMs really work serve as a useful reminder that we're looking at statistical language models, not sentient computers.

Re: Notes on OpenAI's new o1 chain-of-thought models

#442

Post by troupo »

KoolKat23 wrote: post #393
troupo wrote: post #188

Earlier quoted context omitted.

> That's the frustrating thing. LLMs don't materially reduce the set of problems where I'm running against a wall or have trouble finding information. As you step outside regular Stack Overflow questions for top-3 languages, you run into limitations of these predictive models. There's no "reasoning" behind them. They are still, largely, bullshit machines.

you're both on the wrong wavelength. No one has claimed it is better than an expert human yet. Be glad, for now your jobs are safe, why not use it as a tool to boost your productivity, yes, even though you'll get proportionally less use than others in other perhaps less "expert" jobs.

In order for it to boost productivity it needs to answer more than the regular questions for the top-3 languages on Stackoverflow, no?

It often fails even for those questions.

If I need to babysit it for every line of code, it's not a productivity boost.

Re: Notes on OpenAI's new o1 chain-of-thought models

#444

Post by Workaccount2 »

tjoff wrote: post #113
MPSimmons wrote: post #106

Earlier quoted context omitted.

The failure is in how you're using it. I don't mean this as a personal attack, but more to shed light on what's happening. A lot of people use LLMs as a search engine. It makes sense - it's basically a lossy compressed database of everything its ever read, and it generates output that is statistically likely - varying degrees of likeliness depending on the temperature, as well as how many times the particular weights…

And how much data can you give it? I'm not up to date with these things because I haven't found them useful. But with what you said, and previous limitations in how much data they can retain essentially makes them pretty darn useless for that task. Great learning tool on common subjects you don't know, such as learning a new programming-language. Also great for inspiration etc. But that's pretty much it? Don't get me…

People never talk about Gemini, and frankly it's output is often the worst of SOTA models, but it's 2M context window is insane.

You can drop a few textbooks into the context window before you start asking questions. This dramatically improves output quality, however inference does take much much longer at large context lengths.

Re: Notes on OpenAI's new o1 chain-of-thought models

#445

Post by kqr »

lewhoo wrote: post #307
kqr wrote: post #131

Earlier quoted context omitted.

> probabilistic token generators aren’t intelligence Maybe this has been extensively discussed before, but since I've lived under a rock: which parts of intelligence do you think are not representable as conditional probability distributions?

> which parts of intelligence do you think are not representable as conditional probability distributions Maybe I'm wrong here but a lot of our brilliance comes from acting against the statistical consensus. What I mean is, Nicolaus Copernicus probably consumed a lot of knowledge on how the Earth is the center of the universe etc. and probably nothing contradicting that notion. Can a LLM do that ?

It could be "probability of token being useful" rather than "probability of token coming next in training data"!

Re: Notes on OpenAI's new o1 chain-of-thought models

#446

Post by HarHarVeryFunny »

idiotsecant wrote: post #391
berniedurfee wrote: post #381

Earlier quoted context omitted.

I’m starting to think this is an unsolvable problem with LLMs. The very act of “reasoning” requires one to know that they don’t know something. LLMs are giant word Plinko machines. A million monkeys on a million typewriters. LLMs are not interns. LLMs are assumption machines. None of the million monkeys or the collective million monkeys are “reasoning” or are capable of knowing. LLMs are a neat parlor trick and are s…

I think LLMs are definitely on the path to AGI in the same way that the ball bearing was on the path to the internal combustion engine. I think its quite likely that LLMs will perform important functions within the system of an eventual AGI.

We're learning valuable lessons from all modern large-scale (post-AlexNet) NN architectures, transformers included, and NNs (but maybe trained differently) seem a viable approach to implement AGI, so we're making progress ... but maybe LLMs will be more inspiration than part of the (a) final solution.

OTOH, maybe pre-trained LLMs could be used as a hardcoded "reptilian brain" that provides some future AGI with some base capabilities (vs being sold as newborn that needs 20 years of parenting to be useful) that the real learning architecture can then override.

Re: Notes on OpenAI's new o1 chain-of-thought models

#447

Post by energy123 »

bambax wrote: post #281

Near the end, the quote from OpenAI researcher Jason Wei seems damning to me: > Results on AIME and GPQA are really strong, but that doesn’t necessarily translate to something that a user can feel. Even as someone working in science, it’s not easy to find the slice of prompts where GPT-4o fails, o1 does well, and I can grade the answer. But when you do find such prompts, o1 feels totally magical. We all need to find…

You're reading too much into an offhand comment that's more metaphorical in nature.

Re: Notes on OpenAI's new o1 chain-of-thought models

#448

Post by airstrike »

elicksaur wrote: post #72
airstrike wrote: post #60

I've just wasted a few rounds of my weekly o1 ammo by feeding it hard problems I have been working on over the last couple days and for which GPT-4o had failed spectacularly. I suppose I'm to blame for raising my own expectations after the latest PR, but I was pretty disappointed when the answers weren't any better from what I got with the old model. TL;DR It felt less like a new model and way more like one of those…

Can you give an example of one of these problems for context?

One of them was figuring out a recursion issue in a grammar for a markup language I wrote. The other was about traversing a dependency graph and evaluating stale nodes.

Re: Notes on OpenAI's new o1 chain-of-thought models

#449

Post by Workaccount2 »

KoolKat23 wrote: post #316
bambax wrote: post #281

Near the end, the quote from OpenAI researcher Jason Wei seems damning to me: > Results on AIME and GPQA are really strong, but that doesn’t necessarily translate to something that a user can feel. Even as someone working in science, it’s not easy to find the slice of prompts where GPT-4o fails, o1 does well, and I can grade the answer. But when you do find such prompts, o1 feels totally magical. We all need to find…

> Results are "strong" but can't be felt by the user? What does that even mean? This means it often doesn't provide the answer the user is looking for. In my opinion, it's an alignment problem, people are very presumptuous and leave out a lot of detail in their request. Like the "which is bigger - 9.8 or 9.11? question, if you ask "numerically which is bigger - 9.8 or 9.11?" It gets the correct answer, basically it p…

The irony here is that Jason is speaking in the context of LLM development, which he lives and breaths all day.

Reading his comments without framing it in that context makes it come off pretty badly - humans failing to understand what is being said because they don't have context.

Re: Notes on OpenAI's new o1 chain-of-thought models

#450

Post by Workaccount2 »

heyitsguay wrote: post #132
andreilys wrote: post #77

Earlier quoted context omitted.

Check out replit agents, they can make games and apps autonomously now

Practical challenge with a $250 prize: Make a 2D isometric HTML+JS game (dealer's choice on library) in the next 48 hours that satisfies these modest random requirements: A character walks around a big ornate classic library, pulling books from bookshelves looking for a special book that causes a shelf to rotate around and reveal a hidden room and treasure chest. The player can read the books and some are just filler…

The question at the heart of people's anxiety here is: Would you bet that same $250 if AI had 5 years to be able to do it?
Post reply on HN