Notes on OpenAI's new o1 chain-of-thought models • Hacker News Board

Re: Notes on OpenAI's new o1 chain-of-thought models

#61

Post by deisteve » Fri, Sep 13, 2024, 2:48 AM UTC

salomonk_mur wrote: post #59

lukev wrote: post #43

It's interesting to note that there's really two things going on here: 1. A LLM (probably a finetuned GPT-4o) trained specifically to read and emit good chain-of-thought prompts. 2. Runtime code that iteratively re-prompts the model with the chain of thought so far. This sounds like it includes loops, branches and backtracking. This is not "the model", it's regular code invoking the model. Interesting that OpenAI is…

The innovation lies in making the whole loop available to an end user immediately, without them being a programmer. My grandma can build games using ChatGPT now.

what game has she made

Re: Notes on OpenAI's new o1 chain-of-thought models

#62

Post by elicksaur » Fri, Sep 13, 2024, 2:49 AM UTC

salomonk_mur wrote: post #59

lukev wrote: post #43

It's interesting to note that there's really two things going on here: 1. A LLM (probably a finetuned GPT-4o) trained specifically to read and emit good chain-of-thought prompts. 2. Runtime code that iteratively re-prompts the model with the chain of thought so far. This sounds like it includes loops, branches and backtracking. This is not "the model", it's regular code invoking the model. Interesting that OpenAI is…

The innovation lies in making the whole loop available to an end user immediately, without them being a programmer. My grandma can build games using ChatGPT now.

What games have people made with ChatGPT? Do you have an example of a live, deployed game?

Re: Notes on OpenAI's new o1 chain-of-thought models

#63

Post by mediaman » Fri, Sep 13, 2024, 2:50 AM UTC

OutOfHere wrote: post #37

jumploops wrote: post #3

> the idea that I can run a complex prompt and have key details of how that prompt was evaluated hidden from me feels like a big step backwards. As a developer, this is highly concerning, as it makes it much harder to debug where/how the “reasoning” went wrong. The pricing is also silly, because I’m paying for tokens I can’t see. As a user, I don’t really care. LLMs are already magic boxes and I usually only care abo…

> As a user, I don’t really care. Tell me: Just how is it fair for a user to pay for the reasoning tokens without actually seeing them? If they are not shared, the service can bill you anything they want for them!

The simple answer is: I don't care. I'll statistically figure out what the typical total cost per call is from experience, and that's what matters. Who cares if they lie about it, if the model's cost per call fits my budget?

If it starts costing $1 per call, and that's too high, then I just won't use it commercially. Whether it was $1 because they inflated the token count or because it just actually took a lot of tokens to do its reasoning isn't really material to my economic decision.

Re: Notes on OpenAI's new o1 chain-of-thought models

#64

Post by throwup238 » Fri, Sep 13, 2024, 2:50 AM UTC

tptacek wrote: post #57

throwup238 wrote: post #53

Earlier quoted context omitted.

> * To convert Linux kernel subsystems into Python so I can quickly understand them (I'm a C programmer but everyone reads Python faster). Do you have an example chat of this output? Sounds interesting. Do you just dump the C source code into the prompt and ask it to convert to Python?

No, ChatGPT is way cooler than that. It's already read every line of kernel code ever written. I start with a subsystem: the device mapper is a good recent example. I ask things like "explain the linux device mapper. if it was a class in an object-oriented language, what would its interface look like?" and "give me dm_target as a python class". I get stuff like: def linear_ctr(target, argc, argv): print("Constructor…

That's awesome. Have you tried asking it to convert Python (psuedo-ish) code back into C that interfaces with the kernel?

Re: Notes on OpenAI's new o1 chain-of-thought models

#65

Post by Xeamek » Fri, Sep 13, 2024, 2:51 AM UTC

salomonk_mur wrote: post #59

lukev wrote: post #43

It's interesting to note that there's really two things going on here: 1. A LLM (probably a finetuned GPT-4o) trained specifically to read and emit good chain-of-thought prompts. 2. Runtime code that iteratively re-prompts the model with the chain of thought so far. This sounds like it includes loops, branches and backtracking. This is not "the model", it's regular code invoking the model. Interesting that OpenAI is…

The innovation lies in making the whole loop available to an end user immediately, without them being a programmer. My grandma can build games using ChatGPT now.

No she can't, comments likes yours are just made up nonsense that AI hype-mans and investors somehow convinced us are a fair opinions to have.

Re: Notes on OpenAI's new o1 chain-of-thought models

#66

Post by amazingamazing » Fri, Sep 13, 2024, 2:53 AM UTC

salomonk_mur wrote: post #59

lukev wrote: post #43

It's interesting to note that there's really two things going on here: 1. A LLM (probably a finetuned GPT-4o) trained specifically to read and emit good chain-of-thought prompts. 2. Runtime code that iteratively re-prompts the model with the chain of thought so far. This sounds like it includes loops, branches and backtracking. This is not "the model", it's regular code invoking the model. Interesting that OpenAI is…

The innovation lies in making the whole loop available to an end user immediately, without them being a programmer. My grandma can build games using ChatGPT now.

My great aunt literally asked o1 for fantasy football bets and won $1000 on draftkings. This is a gamechanger

Re: Notes on OpenAI's new o1 chain-of-thought models

#67

Post by elicksaur » Fri, Sep 13, 2024, 2:53 AM UTC

OutOfHere wrote: post #37

jumploops wrote: post #3

> the idea that I can run a complex prompt and have key details of how that prompt was evaluated hidden from me feels like a big step backwards. As a developer, this is highly concerning, as it makes it much harder to debug where/how the “reasoning” went wrong. The pricing is also silly, because I’m paying for tokens I can’t see. As a user, I don’t really care. LLMs are already magic boxes and I usually only care abo…

> As a user, I don’t really care. Tell me: Just how is it fair for a user to pay for the reasoning tokens without actually seeing them? If they are not shared, the service can bill you anything they want for them!

The person you are replying to points this out. They make a distinction between developers and users. An end user on a monthly subscription plan doesn’t care about how much compute happens for their chat.

Re: Notes on OpenAI's new o1 chain-of-thought models

#68

Post by devbent » Fri, Sep 13, 2024, 2:55 AM UTC

ldjkfkdsjnv wrote: post #15

OutOfHere wrote: post #12

Earlier quoted context omitted.

> A properly written requirements spec, and an engineer, can do the work of 5. I do not think this will scale. GPT o1 is presumably good for bootstrapping a project using tools that the engineer is not familiar with. The model will struggle to update a sizable codebase, however, with dependencies between the files. Secondly, no matter the size of the codebase and no matter the model used, the engineer still has to re…

I respectfully, but completely disagree. Right now with sonnet 3.5 + cursor ide, I'm not writing that much of my own code at my FAANG job. I am generating a ton, passing in documentation from internal libraries, iterating on the result. Most of the time, I just accept its changes. This is going to rapidly happen. All we need are a few more model releases, not even a step function improvement

Find harder problems to solve.

I can easily make Claude freak out and run into limits. Claude is amazing but it only works at the abstraction level you ask of it, so if you ask it to write code to solve a problem it'll only solve that immediate problem, it doesn't have awareness of any larger refactorings or design improvements that could be made to improve what solution is even possible.

Re: Notes on OpenAI's new o1 chain-of-thought models

#69

Post by throwup238 » Fri, Sep 13, 2024, 2:56 AM UTC

mediaman wrote: post #56

faangguyindia wrote: post #50

Earlier quoted context omitted.

>The o1-preview model still hallucinates non-existing libraries and functions for me, and is quickly wrong about facts that aren't well-represented on the web. It's the usual string of "You're absolutely correct, and I apologize for the oversight in my previous response. [Let me make another guess.]" After that you switch to Claude Soñnet and after sometime it also gets stuck. Problem with LLM is that they are not aw…

One trick that people are using, when using Cursor and specifically Cursor's compose function, is to dump library docs into a text file in your repo, and then @ that doc file when you're asking it to do something involving that library. That seems to eliminate a lot of the issues, though it's not a seamless experience, and it adds another step of having to put the library docs in a text file. Alternatively, cursor ca…

Cursor also has the shadow workspace feature [1] that is supposed to send feedback from linting and language servers to the LLM. I'm not sure whether it's enabled in compose yet though.

[1] https://www.cursor.com/blog/shadow-workspace

Re: Notes on OpenAI's new o1 chain-of-thought models

#70

Post by whynotminot » Fri, Sep 13, 2024, 2:57 AM UTC

famouswaffles wrote: post #4

GaggiX wrote: post #2

I imagine that GPT-5 would be a refined version of this paradigm, probably with omni (multimodal) capabilities added (input and output).

Reports from the Information and the like have been that this is/was being used to generate a lot of synthetic data to train Orion (~GPT-5 Codename).

I'm guessing the true core of this product is still GPT-4, wrapped in whatever new logic they've created to force it through more reasoning iterations.

If o1 was indeed used to create synthetic data to make the upcoming GPT-5, you can perhaps glimpse an interesting level-up process laid out here. GPT-5 could then take over at the heart of a hypothetical o2, yielding a big upgrade. Which would then be leveraged to generate synthetic data to train GPT-6. Which would then form the heart of o3. Etc.