Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Notes on OpenAI's new o1 chain-of-thought models

simonwillison.net

Post reply on HN
Print view View original

581–590 of 659 posts

Re: Notes on OpenAI's new o1 chain-of-thought models

#581

Post by Bluestein »

ethbr1 wrote: post #532
mlsu wrote: post #514

Earlier quoted context omitted.

This is the way. I've had this experience many times: - hey, can you write me a thing that can do "xyz" - sure, here's how we can do "xyz" (gets some small part of the error handling for xyz slightly wrong) - can you add onto this with "abc" - sure. in order to do "abc" we'll need to add "lmn" to our error handling. this also means that you need "ijk" and "qrs" too, and since "lmn" doesn't support "qrs" out of the bo…

> If you don't hit that kill switch, it just keeps doubling down on absurdly complex/incorrect/hallucinatory stuff. If you think about it, that's probably the most difficult problem conversational LLMs need to overcome -- balancing sticking to conversational history vs abandoning it. Humans do this intuitively. But it seems really difficult to simultaneously (a) stick to previous statements sufficiently to avoid seem…

> stick to previous statements sufficiently to avoid seeming ADD in a conveSQUIRREL

:)

Re: Notes on OpenAI's new o1 chain-of-thought models

#582

Post by Bluestein »

sqeaky wrote: post #574
lupire wrote: post #556

Earlier quoted context omitted.

Here's something a human does but an LLM doesn't: If you talk for a while and the facts don't add up and make sense, an intelligent human will notice that, and get upset, and will revisit and dig in and propose experiments and make edits to make all the facts logically consistent. An LLM will just happily go in circles respinning the garbage.

I want to hang out with the humans you've been hanging out with. I know so many people who can't process basic logic or evidence that for my pandemic project a few years I did a year-long podcast about it, even made up a new word describe people who couldn't process evidence "Dysevidentia".

> "Dysevidentia"

This is great.-

Re: Notes on OpenAI's new o1 chain-of-thought models

#583

Post by int_19h »

ryoshu wrote: post #385
williamdclt wrote: post #164

Earlier quoted context omitted.

> Treat it as a naive but intelligent intern That’s the problem: it’s a _terrible_ intern. A good intern will ask clarifying questions, tell me “I don’t know” or “I’m not sure I did it right”. LLMs do none of that, they will take whatever you ask and give a reasonable-sounding output that might be anything between brilliant and nonsense. With an intern, I don’t need to measure how good my prompting is, we’ll usually…

1000% this. LLMs can't say "I don't know" because they don't actually think. I can coach a junior to get better. LLMs will just act like they know what they are doing and give the wrong results to people who aren't practitioners. Good on OAI calling their model Strawberry because of Internet trolls. Reactive vs proactive.

The LLMs absolutely can and do say "I don't know"; I've seen it with both GPT-4 and LLaMA. They don't do it anywhere near as much as they should, yes - likely because their training data doesn't include many examples of that, proportionally - but they are by no means incapable of it.

Re: Notes on OpenAI's new o1 chain-of-thought models

#584

Post by int_19h »

swader999 wrote: post #426
idiotsecant wrote: post #391

Earlier quoted context omitted.

I think LLMs are definitely on the path to AGI in the same way that the ball bearing was on the path to the internal combustion engine. I think its quite likely that LLMs will perform important functions within the system of an eventual AGI.

This may be accurate. I wonder if there's enough energy in the world for this endeavour.

Humans don't require immense amounts of energy to function. The reasons why LLMs do is because we are essentially using brute force as the methodology for making them smarter for the lack of better understanding of how this works. But this then gives us a lot of material to study to figure that part out for future iterations of the concept.

Re: Notes on OpenAI's new o1 chain-of-thought models

#585

Post by kristianp »

jes5199 wrote: post #481
kristianp wrote: post #104

I tried a problem I was looking at recently, to refactor a small rust crate to use one datatype instead of an enum, to help me understand the code better. I found o1-mini made a decent attempt, but couldn't provide error free code. o1-preview was able to provide code that compiled and passed all but the test that is expected to fail, given the change I asked it to make. This is the prompt I gave: simplify this rust l…

I cannot tell from reading what you wrote whether you think it did a good job or not

Thanks for the feedback. I do think it did a good job in the end. I haven't had time to have a good look at the final code o1-preview produced and also my understanding of rust is pretty basic, which I why I didn't say more about the results. I think rust is one of those languages where, if it compiles, you're most of the way there, because of the strong type system. Not as strong as Haskell or Ocaml perhaps.

Re: Notes on OpenAI's new o1 chain-of-thought models

#586

Post by m101 »

helmsb wrote: post #33

I did a few tests and asked it some legal questions. 4o gave me the correct answer immediately. o1 preview gave a much more in depth but completely wrong answer. It took 5 follow ups to get it to recognize that it hallucinated a non-existent law

Perhaps the smaller model used in o1 is over trained on arxiv and code relative to 4o (or undertrained on legal text)

Re: Notes on OpenAI's new o1 chain-of-thought models

#587

Post by HarHarVeryFunny »

throwaway4aday wrote: post #575
HarHarVeryFunny wrote: post #446

Earlier quoted context omitted.

We're learning valuable lessons from all modern large-scale (post-AlexNet) NN architectures, transformers included, and NNs (but maybe trained differently) seem a viable approach to implement AGI, so we're making progress ... but maybe LLMs will be more inspiration than part of the (a) final solution. OTOH, maybe pre-trained LLMs could be used as a hardcoded "reptilian brain" that provides some future AGI with some b…

I would think they'd be more likely to form the language centre of a composite AGI brain. If you read through the known functions of the various areas involved in language[0] they seem to map quite well to the capabilities of transformer based LLMs especially the multi-modal ones. [0] https://en.wikipedia.org/wiki/Language_center

It's not obvious that an LLM - a pre-trained/frozen chunk of predictive statistics - would be amenable to being used as an integral part of an AGI that would necessarily be using a different incremental learning algorithm.

Would the transformer architecture be compatible with the needs of an incremental learning system? It's missing the top down feedback paths (finessed by SGD training) needed to implement prediction-failure driven learning that feature so heavily in our own brain.

This is why I could more see a potential role for a pre-trained LLM as a separate primitive subsystem to be overidden, or maybe (more likely) we'll just pre-expose an AGI brain to 20 years of sped-up life experience and not try to import an LLM to be any part of it!

Re: Notes on OpenAI's new o1 chain-of-thought models

#588

Post by m101 »

HarHarVeryFunny wrote: post #451

I think Rich Sutton's bitter lesson will prove to apply here, and what we really need to advance machine learning capabilities are more general and powerful models capable of learning for themselves - better able to extract and use knowledge from the firehose of data available from the real world (ultimately via some form of closed-loop deployment where they can act and incrementally learn from their own actions). Wh…

It works like our own minds in that we also think, test, go back, try again. This doesn't seem like a failing but just a recognition that thought can proceed in that way.

Re: Notes on OpenAI's new o1 chain-of-thought models

#589

Post by fragmede »

raincole wrote: post #571
ActionHank wrote: post #461

Earlier quoted context omitted.

The stupidest thing about ai and automation is that they are trying to target it at large corporations looking to cut down on jobs or 10x productivity when all anyone actually wants is a robot to do their laundry and dishes.

Because a robot that do everyone's laundry is much more closer to AGI than ChatGPT. I'm dead serious.

Not really. You don't need to move wet clothes from the first machine to a second machine if you get one machine that does both jobs. That's very much not AGI. The second job, of taking dry crumpled clothes and folding them, also doesn't need an artificial general intelligence. It's very computationally expensive (as evidenced by the speed of https://pantor.github.io/speedfolding/, out of UC Berkeley) and a hard robotics question, but it's also very fixed function.

Taking the clothes out of the combined washer dryer machine, my laundry folding robot isn't suddenly going to need to come up with a creative answer to a question I have about politics in order to fold the laundry, or come up with a new way to organize my board game collection, or reason about how to refactor some code. There are no logical leaps of reasoning or deep thinking required. My laundry folding robot doesn't need to be creative in order to fold laundry, just application of some very complex algorithms, some of which have yet to be discovered.

Re: Notes on OpenAI's new o1 chain-of-thought models

#590

Post by KoolKat23 »

dartos wrote: post #552
KoolKat23 wrote: post #419

Earlier quoted context omitted.

That is indeed cynical haha. A very simple observation, our brains are vastly more efficient. Obtaining vastly better outcomes from lesser input. This evidence means there's plenty of room for improvement without a need to go looking for more data. Short term gain versus long term gain like you say, shareholder return. More efficiency means more practical/useful applications and lower cost as opposed to bigger model…

That’s assuming that LLMs act like brains at all. They don’t. Especially not with transformers.

Says who? At a fundamental level
Post reply on HN