I did a few tests and asked it some legal questions. 4o gave me the correct answer immediately. o1 preview gave a much more in depth but completely wrong answer. It took 5 follow ups to get it to recognize that it hallucinated a non-existent law
Notes on OpenAI's new o1 chain-of-thought models
71–80 of 659 posts
Re: Notes on OpenAI's new o1 chain-of-thought models
#72I've just wasted a few rounds of my weekly o1 ammo by feeding it hard problems I have been working on over the last couple days and for which GPT-4o had failed spectacularly. I suppose I'm to blame for raising my own expectations after the latest PR, but I was pretty disappointed when the answers weren't any better from what I got with the old model. TL;DR It felt less like a new model and way more like one of those…
Re: Notes on OpenAI's new o1 chain-of-thought models
#73It's interesting to note that there's really two things going on here: 1. A LLM (probably a finetuned GPT-4o) trained specifically to read and emit good chain-of-thought prompts. 2. Runtime code that iteratively re-prompts the model with the chain of thought so far. This sounds like it includes loops, branches and backtracking. This is not "the model", it's regular code invoking the model. Interesting that OpenAI is…
Re: Notes on OpenAI's new o1 chain-of-thought models
#74Earlier quoted context omitted.
No, ChatGPT is way cooler than that. It's already read every line of kernel code ever written. I start with a subsystem: the device mapper is a good recent example. I ask things like "explain the linux device mapper. if it was a class in an object-oriented language, what would its interface look like?" and "give me dm_target as a python class". I get stuff like: def linear_ctr(target, argc, argv): print("Constructor…
That's awesome. Have you tried asking it to convert Python (psuedo-ish) code back into C that interfaces with the kernel?
I do feel like someone who skipped like 8 iPhone models (cross-referencing, EIEIO, lsp-mode, code explorers, tree-sitter) and just got an iPhone 16. Like, nothing that came before this for code comprehension really matters all that much?
Re: Notes on OpenAI's new o1 chain-of-thought models
#75Earlier quoted context omitted.
I honestly can’t believe this is the hyped up “strawberry” everyone was claiming is pretty much AGI. Senior employees leaving due to its powers being so extreme I’m in the “probabilistic token generators aren’t intelligence” camp so I don’t actually believe in AGI, but I’ll be honest the never ending rumors / chatter almost got to me Remember, this is the model some media outlet reported recently that is so powerful…
I mean, considering how many tokens their example prompt consumed, I wouldn't be surprised if it costs ~$2k/month/user to run
Re: Notes on OpenAI's new o1 chain-of-thought models
#76Earlier quoted context omitted.
The innovation lies in making the whole loop available to an end user immediately, without them being a programmer. My grandma can build games using ChatGPT now.
No she can't, comments likes yours are just made up nonsense that AI hype-mans and investors somehow convinced us are a fair opinions to have.
Re: Notes on OpenAI's new o1 chain-of-thought models
#77Earlier quoted context omitted.
The innovation lies in making the whole loop available to an end user immediately, without them being a programmer. My grandma can build games using ChatGPT now.
No she can't, comments likes yours are just made up nonsense that AI hype-mans and investors somehow convinced us are a fair opinions to have.
Re: Notes on OpenAI's new o1 chain-of-thought models
#78Earlier quoted context omitted.
I dunno man. I just spent a couple hours trying to get it to write functioning code to read from my RTSP stream, detect if my kid is playing piano, and send the result to HomeAssistant. It did not succeed.
How many hours without it?
Stuck in loops, correct their mistakes with worse mistakes, hallucinating things that don’t exist and being unable to correct.
Working on my own, I have the confidence that I know I can make incremental forward progress on a problem. That’s much preferable.
Re: Notes on OpenAI's new o1 chain-of-thought models
#79Re: Notes on OpenAI's new o1 chain-of-thought models
#80Earlier quoted context omitted.
No she can't, comments likes yours are just made up nonsense that AI hype-mans and investors somehow convinced us are a fair opinions to have.
Check out replit agents, they can make games and apps autonomously now