Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Notes on OpenAI's new o1 chain-of-thought models

simonwillison.net

Post reply on HN
Print view View original

131–140 of 659 posts

Re: Notes on OpenAI's new o1 chain-of-thought models

#131

Post by kqr »

shmatt wrote: post #51
layer8 wrote: post #18

The o1-preview model still hallucinates non-existing libraries and functions for me, and is quickly wrong about facts that aren't well-represented on the web. It's the usual string of "You're absolutely correct, and I apologize for the oversight in my previous response. [Let me make another guess.]" While the reasoning may have been improved, this doesn't solve the problem of the model having no way to assess if what…

I honestly can’t believe this is the hyped up “strawberry” everyone was claiming is pretty much AGI. Senior employees leaving due to its powers being so extreme I’m in the “probabilistic token generators aren’t intelligence” camp so I don’t actually believe in AGI, but I’ll be honest the never ending rumors / chatter almost got to me Remember, this is the model some media outlet reported recently that is so powerful…

> probabilistic token generators aren’t intelligence

Maybe this has been extensively discussed before, but since I've lived under a rock: which parts of intelligence do you think are not representable as conditional probability distributions?

Re: Notes on OpenAI's new o1 chain-of-thought models

#132

Post by heyitsguay »

andreilys wrote: post #77
Xeamek wrote: post #65

Earlier quoted context omitted.

No she can't, comments likes yours are just made up nonsense that AI hype-mans and investors somehow convinced us are a fair opinions to have.

Check out replit agents, they can make games and apps autonomously now

Practical challenge with a $250 prize: Make a 2D isometric HTML+JS game (dealer's choice on library) in the next 48 hours that satisfies these modest random requirements:

A character walks around a big ornate classic library, pulling books from bookshelves looking for a special book that causes a shelf to rotate around and reveal a hidden room and treasure chest. The player can read the books and some are just filler but some have clues about the special book. If this can be done with art, animations, sound, UI, the usual stuff, I'll believe the parent poster's claim to be true.

As someone using LLM-based workflows daily to assist with personal and professional projects, I'll wager $250 that this is not possible.

Re: Notes on OpenAI's new o1 chain-of-thought models

#135

Post by gsinclair »

kylebenzle wrote: post #119
MPSimmons wrote: post #106

Earlier quoted context omitted.

The failure is in how you're using it. I don't mean this as a personal attack, but more to shed light on what's happening. A lot of people use LLMs as a search engine. It makes sense - it's basically a lossy compressed database of everything its ever read, and it generates output that is statistically likely - varying degrees of likeliness depending on the temperature, as well as how many times the particular weights…

Perfectly well put! We should change the name from "AI" (which it is not) to something like, "lossy compressed databases".

That abbreviates to LCD. If we could make it LSD somehow, that would help to explain the hallucinations.

Re: Notes on OpenAI's new o1 chain-of-thought models

#136

Post by ben_w »

tsunamifury wrote: post #41
mensetmanusman wrote: post #38

Earlier quoted context omitted.

LLMs are not for expanding the sphere of human knowledge, but for speeding up auto-correct of higher order processing to help you more quickly reach the shell of the sphere and make progress with your own mind :)

Definitely. When we talk about being skilled in a T shape LLMs are all about spreading your top of T and not making the bottom go deeper.

Indeed, not much more depth — though even Terence Tao reported useful results from an earlier version, so perhaps the breadth is a depth all of it's own: https://mathstodon.xyz/@tao/110601051375142142

I think of it as making the top bar of the T thicker, but yes, you're right, it also spreads it much wider.

Re: Notes on OpenAI's new o1 chain-of-thought models

#138

Post by colordrops »

cma wrote: post #39
colordrops wrote: post #36

Earlier quoted context omitted.

The answer is yes if you are willing to code it. OpenAI supports tool calls. Even if it didn't you could just make multiple calls to their API and submit the result of the code execution yourself.

The intermediate CoT results aren't in the API.

I may be mistaken but I don't believe the first version of the comment I replied to mentioned intermediate CoT results.

Re: Notes on OpenAI's new o1 chain-of-thought models

#139

Post by [deleted] »

jumploops wrote: post #3

> the idea that I can run a complex prompt and have key details of how that prompt was evaluated hidden from me feels like a big step backwards. As a developer, this is highly concerning, as it makes it much harder to debug where/how the “reasoning” went wrong. The pricing is also silly, because I’m paying for tokens I can’t see. As a user, I don’t really care. LLMs are already magic boxes and I usually only care abo…

[deleted]

Re: Notes on OpenAI's new o1 chain-of-thought models

#140

Post by MikeTheGreat »

gsinclair wrote: post #135
kylebenzle wrote: post #119

Earlier quoted context omitted.

Perfectly well put! We should change the name from "AI" (which it is not) to something like, "lossy compressed databases".

That abbreviates to LCD. If we could make it LSD somehow, that would help to explain the hallucinations.

Lossy Stochastic Database?
Post reply on HN