Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Notes on OpenAI's new o1 chain-of-thought models

simonwillison.net

Post reply on HN
Print view View original

551–560 of 659 posts

Re: Notes on OpenAI's new o1 chain-of-thought models

#551

Post by hmottestad »

roywiggins wrote: post #434
lucubratory wrote: post #182

Earlier quoted context omitted.

Yeah, I think what a lot of people miss about these sort of gotchas are that most of them were invented explicitly to gotcha humans, who regularly get got by them. This is not a failure mode unique to LLMs.

One that trips up LLMs in ways that wouldn't trip up humans is the chicken, fox and grain puzzle but with just the chicken. They tend to insist that the chicken be taken across the river, then back, then across again, for no reason other than the solution to the classic puzzle requires several crossings. No human would do that, by the time you've had the chicken across then even the most unobservant human would reali…

Here you go: https://chatgpt.com/share/66e48de6-4898-800e-9aba-598a57d27f...

Seemed to handle it just fine.

Kinda a waste of a perfectly good LLM if you ask me. I've mostly been using it as a coding assistant today and it's been absolutely great. Nothing too advanced yet, mostly mundane changes that I got bored of having to make myself. Been giving it very detailed and clear instructions, like I would to a Junior developer, and not giving it too many steps at once. Only issue I've run into is that it's fairly slow and that breaks my coding flow.

Re: Notes on OpenAI's new o1 chain-of-thought models

#552

Post by dartos »

KoolKat23 wrote: post #419
Culonavirus wrote: post #358

Earlier quoted context omitted.

A cynical way to look at it is that we're pretty close to the ultimate limits of what LLMs can do and now the stake holders are looking at novel ways of using what they have instead of pouring everything into novel models. We're several years into the AI revolution (some call it a bubble) and Nvidia is still pretty much the only company that makes bank on it. Other than that it's all investment driven "growth". And a…

That is indeed cynical haha. A very simple observation, our brains are vastly more efficient. Obtaining vastly better outcomes from lesser input. This evidence means there's plenty of room for improvement without a need to go looking for more data. Short term gain versus long term gain like you say, shareholder return. More efficiency means more practical/useful applications and lower cost as opposed to bigger model…

That’s assuming that LLMs act like brains at all.

They don’t.

Especially not with transformers.

Re: Notes on OpenAI's new o1 chain-of-thought models

#553

Post by koe123 »

richerram wrote: post #541
williamdclt wrote: post #164

Earlier quoted context omitted.

> Treat it as a naive but intelligent intern That’s the problem: it’s a _terrible_ intern. A good intern will ask clarifying questions, tell me “I don’t know” or “I’m not sure I did it right”. LLMs do none of that, they will take whatever you ask and give a reasonable-sounding output that might be anything between brilliant and nonsense. With an intern, I don’t need to measure how good my prompting is, we’ll usually…

I think this is the main issue with these tools... what people are expecting of them. We have swallowed the pill that LLMs are supposed to be AGI and all that mumbo jumbo, when they are just great tools and as such one needs to learn to use the tool the way it works and make the best of it, nobody is trying to hammer a nail with a broom and blaming the broom for not being a hammer...

I completely agree.

To me the discussion here reads a little like: “Hah. See? It cant do everything!”. It makes me wonder if the goal is to convince each other that: yes, indeed, humans are not yet replaced.

It’s next token regression, of course it can’t truely introspect. That being said LLMs are amazing tools and o1 is yet another incremental improvement and I welcome it!

Re: Notes on OpenAI's new o1 chain-of-thought models

#554

Post by golergka »

lucianbr wrote: post #294
mediaman wrote: post #63

Earlier quoted context omitted.

The simple answer is: I don't care. I'll statistically figure out what the typical total cost per call is from experience, and that's what matters. Who cares if they lie about it, if the model's cost per call fits my budget? If it starts costing $1 per call, and that's too high, then I just won't use it commercially. Whether it was $1 because they inflated the token count or because it just actually took a lot of tok…

The thing is it might increase in cost after you've decided to use it commercially, and have invested a lot of time and resources in it. Now it's very hard to move to something else, but very easy for OpenAI to increase your cost arbitrarily. The statistics you made are not binding for them.

This is experimental, frontier stuff, obviously it comes with risks. Building on GPT-4 in March of 2023 was like that as well, but now you can easily switch between a few models of comparable quality made by different companies (yay capitalism and free markets!). You can risk and use just released stuff right now, or, most likely, come back in 6-12 months (probably earlier) and get several different providers with very similar APIs.

Re: Notes on OpenAI's new o1 chain-of-thought models

#555

Post by troupo »

TeMPOraL wrote: post #522
troupo wrote: post #442

Earlier quoted context omitted.

In order for it to boost productivity it needs to answer more than the regular questions for the top-3 languages on Stackoverflow, no? It often fails even for those questions. If I need to babysit it for every line of code, it's not a productivity boost.

If you need to babysit it for every line of code, you're either a superhuman coder, working in some obscure alien language, or just using the LLM wrong.

No. I'm just using for simple things like "Help me with the Elixir code" or "I need to list Bonjour services using Swift".

It's shit across the whole "AI" spectrum from ChatGPT to Copilot to Cursor aka Claude.

I'm not even talking about code I work with at work, it's just side projects.

As for "using LLMs wrong", using them "right" is literally babysitting their output and spending a lot of time trying to reverse-engineer their behavior with increasingly inane prompts.

Edit: I mean, look at this ridiculousness: https://cursor.directory/

Re: Notes on OpenAI's new o1 chain-of-thought models

#556

Post by lupire »

ethbr1 wrote: post #532
mlsu wrote: post #514

Earlier quoted context omitted.

This is the way. I've had this experience many times: - hey, can you write me a thing that can do "xyz" - sure, here's how we can do "xyz" (gets some small part of the error handling for xyz slightly wrong) - can you add onto this with "abc" - sure. in order to do "abc" we'll need to add "lmn" to our error handling. this also means that you need "ijk" and "qrs" too, and since "lmn" doesn't support "qrs" out of the bo…

> If you don't hit that kill switch, it just keeps doubling down on absurdly complex/incorrect/hallucinatory stuff. If you think about it, that's probably the most difficult problem conversational LLMs need to overcome -- balancing sticking to conversational history vs abandoning it. Humans do this intuitively. But it seems really difficult to simultaneously (a) stick to previous statements sufficiently to avoid seem…

Here's something a human does but an LLM doesn't:

If you talk for a while and the facts don't add up and make sense, an intelligent human will notice that, and get upset, and will revisit and dig in and propose experiments and make edits to make all the facts logically consistent. An LLM will just happily go in circles respinning the garbage.

Re: Notes on OpenAI's new o1 chain-of-thought models

#557

Post by elicksaur »

cdelsolar wrote: post #107
elicksaur wrote: post #62

Earlier quoted context omitted.

What games have people made with ChatGPT? Do you have an example of a live, deployed game?

Yes, a gazillion of them. Someone in a scrabble Facebook group made this entirely with ChatGPT: https://aboocher.github.io/scrabble/ingpractice.html

Look, I get the societal development that you can input narrative text and the code for this pops out is super neat.

But trying to be fair here, anyone would call this incomplete, right?

There are several obvious bugs in styling and interaction.

This example is exactly what I was expecting. An ephemeral, simple-yet-buggy single page that’s barely a game in common understanding.

That person, while maybe not actively programming things, does appear to have forked several repos on GitHub a decade ago. I would say that’s above the level of technical competence implied by the “my grandma” phrasing of the OP.

Re: Notes on OpenAI's new o1 chain-of-thought models

#558

Post by OutOfHere »

golergka wrote: post #554
lucianbr wrote: post #294

Earlier quoted context omitted.

The thing is it might increase in cost after you've decided to use it commercially, and have invested a lot of time and resources in it. Now it's very hard to move to something else, but very easy for OpenAI to increase your cost arbitrarily. The statistics you made are not binding for them.

This is experimental, frontier stuff, obviously it comes with risks. Building on GPT-4 in March of 2023 was like that as well, but now you can easily switch between a few models of comparable quality made by different companies (yay capitalism and free markets!). You can risk and use just released stuff right now, or, most likely, come back in 6-12 months (probably earlier) and get several different providers with ve…

Everything that OpenAI does with LLMs has already been done and validated in the open source community well before OpenAI gets around to it. OpenAI is not an innovator. simbianai/taskgen on github is an example of one such project, although there are others too that don't come to mind right now.

As such, I would never call their work "frontier stuff", but they do bring it to the masses with their commercial service.

Re: Notes on OpenAI's new o1 chain-of-thought models

#559

Post by lupire »

ryoshu wrote: post #407
bartread wrote: post #394

Earlier quoted context omitted.

I get a lot of value out of ChatGPT but I also, fairly frequently, run into issues here. The real danger zones are areas that lie at or just beyond the edges of my own knowledge in a particular area. I'd say that most of my work use of ChatGPT does in fact save me time but, every so often, ChatGPT can still bullshit convincingly enough to waste an hour or two for me. The balance is still in its favour, but you have t…

Agreed, but the problem is if these things replace practitioners (what every MBA wants them to do), it's going to wreck the industry. Or maybe we'll get paid $$$$ to fix the problems they cause. GPT-4 introduced me to window functions in SQL (haven't written raw SQL in over a decade). But I'm experienced enough to look at window functions and compare them to subqueries and run some tests through the query planner to…

Optimizing a query is certainly something the machine (not necessarily the LLM part) can do better than the human, for 99.9% of situations and people.

PostgreSQL developers are oposed to query execution hints, because if a human knows a better way to execute a query, the devs want to put that knowledge into the planner.

Re: Notes on OpenAI's new o1 chain-of-thought models

#560

Post by lupire »

accountnum wrote: post #325
rdtsc wrote: post #253

Earlier quoted context omitted.

Fluency is a necessary but not the only prerequisite. To be able to answer a trick question, it’s first necessary to understand the question.

No, it's necessary to either know that it's a trick question or to have a feeling that it is based on context. The entire point of a question like that is to trick your understanding. You're tricking the model because it has seen this specific trick question a million times and shortcuts to its memorized solution. Ask it literally any other question, it can be as subtle as you want it to be, and the model will pick u…

Some people solve trick questions easily simply because they are slow thinkers who pay attention to every question, even non-trick questions, and don't fast-path the answer based on its similarity to a past question.

Interestingly, people who make bad fast-path answers often call these people stupid.

Post reply on HN