Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Notes on OpenAI's new o1 chain-of-thought models

simonwillison.net

Post reply on HN
Print view View original

511–520 of 659 posts

Re: Notes on OpenAI's new o1 chain-of-thought models

#511

Post by 015a »

littlestymaar wrote: post #495
deegles wrote: post #482

I was thinking about what "actual" AI would be for me and it would be something that could answer questions like "tell me every time Nicolas Cage has blinked while on camera in one of his movies". Sure, that is a contrived question, but I expect an "AI" to be capable pf obtaining every movie, watching them frame-by-frame, and getting an accurate count. All in a few seconds. Current models (any LLM) cannot do that and…

> All in a few seconds That part is unrealistic: even just loading in RAM and decoding all movies Nicolas Cage appears in would take much more than a few seconds unless you thrown an insane amount of compute at the job. That being said, the current LLM tech is probably enough to help you implement a program that parses IMDB to get the list of all Nicolas Cage movie, then download it on thepiratebay and then implement…

So what you're saying is, LLMs are good enough to do something that humans are already capable of doing, in a timeframe that a human would be reasonably capable of doing it in, and its unrealistic to believe that LLMs will ever be able to do something truly superhuman. Got it :+1:

Re: Notes on OpenAI's new o1 chain-of-thought models

#512

Post by skybrian »

prog_1 wrote: post #401
aubanel wrote: post #296

Earlier quoted context omitted.

He's speaking about his objective to make ever stronger LLMs: so for this his secondary objective is to measure their real performance. The human preference is not that good of a proxy measurement: for instance, it can be gamed by making the model more assertive, causing the human error-spotting ability to decrease a lot [0]. So what he's really saying is that non-rigorous human vibe checks (like those LMSys Chatbot…

ie when you cant beat them, make new metrics and you can absolutely evaluate how smart someone is in a 2min casual conversation. You wont be able to tell how well they are in some niche topic, but %insert something about different flavors of intelligence and how they do not equate do subject matter expertise%

It’s a common pattern that AI benchmarks get too easy, so they make new ones that are harder.

Re: Notes on OpenAI's new o1 chain-of-thought models

#513

Post by semi-extrinsic »

shmatt wrote: post #379
synarchefriend wrote: post #335

Earlier quoted context omitted.

"Senior employees leaving due to its powers being so extreme" This never happened. No one said it happened. "the model some media outlet reported recently that is so powerful OAI is considering charging $2k/month for" The Information reported someone at a meeting suggested this for future models, not specifically Strawberry, and that it would probably not actually be that high.

Elon Musk and Ilya Sutskever Have Warned About OpenAI’s ‘Strawberry’ Jul 15, 2024 — Sutskever himself had reportedly begun to worry about the project's technology, as did OpenAI employees working on A.I. safety at the time. https://observer.com/2024/07/openai-employees-concerns-straw... And I’m ignoring the hundreds of Reddit articles speculating every time someone at OAI leaves And of course that $2000 article was s…

My hypothesis about these people who are afraid of AI, is that they have tricked themselves into believing they are in their current position of influence due to their own intelligence (as opposed to luck, connections, etc.)

Then they drink the marketing koolaid, and it follows naturally that they worry an AI system can obtain similar positions of influence.

Re: Notes on OpenAI's new o1 chain-of-thought models

#514

Post by mlsu »

noisy_boy wrote: post #432
williamdclt wrote: post #164

Earlier quoted context omitted.

> Treat it as a naive but intelligent intern That’s the problem: it’s a _terrible_ intern. A good intern will ask clarifying questions, tell me “I don’t know” or “I’m not sure I did it right”. LLMs do none of that, they will take whatever you ask and give a reasonable-sounding output that might be anything between brilliant and nonsense. With an intern, I don’t need to measure how good my prompting is, we’ll usually…

I feel like it almost always starts well, given the full picture, but then for non-trivial stuff, gets stuck towards the end. The longer the conversation goes, the more wheel-spinning occurs and before you know it, you have spent an hour chasing that last-mile-connectivity. For complex questions, I now only use it to get the broad picture and once the output is good enough to be a foundation, I build the rest of it m…

This is the way.

I've had this experience many times:

- hey, can you write me a thing that can do "xyz"

- sure, here's how we can do "xyz" (gets some small part of the error handling for xyz slightly wrong)

- can you add onto this with "abc"

- sure. in order to do "abc" we'll need to add "lmn" to our error handling. this also means that you need "ijk" and "qrs" too, and since "lmn" doesn't support "qrs" out of the box, we'll also need a design solution to bridge the two. Let me spend 600 more tokens sketching that out.

- what if you just use the language's built in feature here in "xyz"? does't that mean we can do it with just one line of code?

- yes, you're absolutely right. I'm sorry for making this over complicated.

If you don't hit that kill switch, it just keeps doubling down on absurdly complex/incorrect/hallucinatory stuff. Even one small error early in the chain propagates. That's why I end up very frequently restarting conversations in a new chat or re-write my chat questions to remove bad stuff from the context. Without the ability to do that, it's nearly worthless. It's also why I think we'll be seeing absurdly, wildly wrong chains of thought coming out of o1. Because "thinking" for 20s may well cause it to just go totally off the rails half the time.

Re: Notes on OpenAI's new o1 chain-of-thought models

#515

Post by jprete »

Spivak wrote: post #485
ActionHank wrote: post #461

Earlier quoted context omitted.

The stupidest thing about ai and automation is that they are trying to target it at large corporations looking to cut down on jobs or 10x productivity when all anyone actually wants is a robot to do their laundry and dishes.

You're describing a dish-washer and washing-machine.

The GP is almost certainly describing a robot that can move dirty stuff into the machines, run them, and put away the clean stuff afterwards.

Re: Notes on OpenAI's new o1 chain-of-thought models

#516

Post by jfengel »

accountnum wrote: post #331
grey-area wrote: post #199

Earlier quoted context omitted.

The 'riddle': A woman and her son are in a car accident. The woman is sadly killed. The boy is rushed to hospital. When the doctor sees the boy he says "I can't operate on this child, he is my son". How is this possible? GPT Answer: The doctor is the boy's mother Real Answer: Boy = Son, Woman = Mother (and her son), Doctor = Father (he says...he is my son) This is not in fact a riddle (though presented as one) and th…

It literally is a riddle, just as the original one was, because it tries to use your expectations of the world against you. The entire point of the original, which a lot of people fell for, was to expose expectations of gender roles leading to a supposed contradiction that didn't exist. You are now asking a modified question to a model that has seen the unmodified one millions of times. The model has an expectation o…

Phrased as it is, it deliberately gives away the answer by using the pronoun "he" for the doctor. The original deliberately obfuscates it by avoiding pronouns.

So it doesn't take an understanding of gender roles, just grammar.

Re: Notes on OpenAI's new o1 chain-of-thought models

#517

Post by noisy_boy »

mlsu wrote: post #514
noisy_boy wrote: post #432

Earlier quoted context omitted.

I feel like it almost always starts well, given the full picture, but then for non-trivial stuff, gets stuck towards the end. The longer the conversation goes, the more wheel-spinning occurs and before you know it, you have spent an hour chasing that last-mile-connectivity. For complex questions, I now only use it to get the broad picture and once the output is good enough to be a foundation, I build the rest of it m…

This is the way. I've had this experience many times: - hey, can you write me a thing that can do "xyz" - sure, here's how we can do "xyz" (gets some small part of the error handling for xyz slightly wrong) - can you add onto this with "abc" - sure. in order to do "abc" we'll need to add "lmn" to our error handling. this also means that you need "ijk" and "qrs" too, and since "lmn" doesn't support "qrs" out of the bo…

> That's why I end up very frequently restarting conversations in a new chat or re-write my chat questions to remove bad stuff from the context.

Me too - open new chat and start by copy/pasting the "last-known-good-state". OpenAI can introduce a "new-chat-from-here" feature :)

Re: Notes on OpenAI's new o1 chain-of-thought models

#518

Post by golergka »

MPSimmons wrote: post #106
layer8 wrote: post #18

The o1-preview model still hallucinates non-existing libraries and functions for me, and is quickly wrong about facts that aren't well-represented on the web. It's the usual string of "You're absolutely correct, and I apologize for the oversight in my previous response. [Let me make another guess.]" While the reasoning may have been improved, this doesn't solve the problem of the model having no way to assess if what…

The failure is in how you're using it. I don't mean this as a personal attack, but more to shed light on what's happening. A lot of people use LLMs as a search engine. It makes sense - it's basically a lossy compressed database of everything its ever read, and it generates output that is statistically likely - varying degrees of likeliness depending on the temperature, as well as how many times the particular weights…

> A lot of people use LLMs as a search engine.

GPT-4o is wonderful as a search engine if you tell it to google things before answering (even though it uses bing).

Re: Notes on OpenAI's new o1 chain-of-thought models

#519

Post by littlestymaar »

015a wrote: post #511
littlestymaar wrote: post #495

Earlier quoted context omitted.

> All in a few seconds That part is unrealistic: even just loading in RAM and decoding all movies Nicolas Cage appears in would take much more than a few seconds unless you thrown an insane amount of compute at the job. That being said, the current LLM tech is probably enough to help you implement a program that parses IMDB to get the list of all Nicolas Cage movie, then download it on thepiratebay and then implement…

So what you're saying is, LLMs are good enough to do something that humans are already capable of doing, in a timeframe that a human would be reasonably capable of doing it in, and its unrealistic to believe that LLMs will ever be able to do something truly superhuman. Got it :+1:

Being able to do “stuff a human is capable of doing” used to be the definition of “artificial intelligence” and until very recently it was seen as a dream that may never happen. And it hasn't completely happened yet BTW, there are still plenty of trivial stuff LLM can't do just because there's no available training data for that. Also their ability to do “reasoning” or few-shot-learning is overhyped (even if impressive).

If your definition of AI has become “superhuman intelligence” then it's definitely moving goalposts. And regaarding my initial remark, AI isn't going to do “faster than the speed of light” MPEG decoding ever, all physical limits apply to it.

Post reply on HN