Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Notes on OpenAI's new o1 chain-of-thought models

simonwillison.net

Post reply on HN
Print view View original

571–580 of 659 posts

Re: Notes on OpenAI's new o1 chain-of-thought models

#571

Post by raincole »

ActionHank wrote: post #461
bambax wrote: post #281

Near the end, the quote from OpenAI researcher Jason Wei seems damning to me: > Results on AIME and GPQA are really strong, but that doesn’t necessarily translate to something that a user can feel. Even as someone working in science, it’s not easy to find the slice of prompts where GPT-4o fails, o1 does well, and I can grade the answer. But when you do find such prompts, o1 feels totally magical. We all need to find…

The stupidest thing about ai and automation is that they are trying to target it at large corporations looking to cut down on jobs or 10x productivity when all anyone actually wants is a robot to do their laundry and dishes.

Because a robot that do everyone's laundry is much more closer to AGI than ChatGPT. I'm dead serious.

Re: Notes on OpenAI's new o1 chain-of-thought models

#572

Post by sqeaky »

jasondigitized wrote: post #564
williamdclt wrote: post #164

Earlier quoted context omitted.

> Treat it as a naive but intelligent intern That’s the problem: it’s a _terrible_ intern. A good intern will ask clarifying questions, tell me “I don’t know” or “I’m not sure I did it right”. LLMs do none of that, they will take whatever you ask and give a reasonable-sounding output that might be anything between brilliant and nonsense. With an intern, I don’t need to measure how good my prompting is, we’ll usually…

Have you ever worked with an intern? They have personalities and expectations that need to be managed. They get sick. The get tired. They want to punch you if you treat them like a 24-7 bird dog. It's so much easier to not let perfect be the enemy of the good and just rapid fire ALL day at a LLM for any and everything I need help with. You can also just not use the LLM. Interns need to be 'fed' work or the ROI ends u…

I think listing off a set of pros and cons for interns and LLMs misses the point, they seem like categorically different kinds of intelligence.

Re: Notes on OpenAI's new o1 chain-of-thought models

#573

Post by fragmede »

raincole wrote: post #563
ramraj07 wrote: post #112

Earlier quoted context omitted.

Many of my PhD and post doc colleagues who emigrated from Korea, China and India who didn’t have English as the medium of instruction would struggle with this question. They only recover when you give them a hint. They’re some of the smartest people in general. If you try to stop stumping these models with trick questions and ask it straightforward reasoning systems it is extremely performant (O1 is definitely a step…

I think you have particularly dumb colleagues then. If you post this question to an average STEM PhD in China (not even from China. In China) they'll get it right. This question is the "unmisleading" version of a very common misleading question about sexism. ChatGPT learned the original, misleading version too well that it can't answer the unmisleading version. Humans who don't have the original version ingrained in…

> it can't answer the unmisleading version.

Yes it can: https://chatgpt.com/share/66e3601f-4bec-8009-ac0c-57bfa4f059...

Re: Notes on OpenAI's new o1 chain-of-thought models

#574

Post by sqeaky »

lupire wrote: post #556
ethbr1 wrote: post #532

Earlier quoted context omitted.

> If you don't hit that kill switch, it just keeps doubling down on absurdly complex/incorrect/hallucinatory stuff. If you think about it, that's probably the most difficult problem conversational LLMs need to overcome -- balancing sticking to conversational history vs abandoning it. Humans do this intuitively. But it seems really difficult to simultaneously (a) stick to previous statements sufficiently to avoid seem…

Here's something a human does but an LLM doesn't: If you talk for a while and the facts don't add up and make sense, an intelligent human will notice that, and get upset, and will revisit and dig in and propose experiments and make edits to make all the facts logically consistent. An LLM will just happily go in circles respinning the garbage.

I want to hang out with the humans you've been hanging out with. I know so many people who can't process basic logic or evidence that for my pandemic project a few years I did a year-long podcast about it, even made up a new word describe people who couldn't process evidence "Dysevidentia".

Re: Notes on OpenAI's new o1 chain-of-thought models

#575

Post by throwaway4aday »

HarHarVeryFunny wrote: post #446
idiotsecant wrote: post #391

Earlier quoted context omitted.

I think LLMs are definitely on the path to AGI in the same way that the ball bearing was on the path to the internal combustion engine. I think its quite likely that LLMs will perform important functions within the system of an eventual AGI.

We're learning valuable lessons from all modern large-scale (post-AlexNet) NN architectures, transformers included, and NNs (but maybe trained differently) seem a viable approach to implement AGI, so we're making progress ... but maybe LLMs will be more inspiration than part of the (a) final solution. OTOH, maybe pre-trained LLMs could be used as a hardcoded "reptilian brain" that provides some future AGI with some b…

I would think they'd be more likely to form the language centre of a composite AGI brain. If you read through the known functions of the various areas involved in language[0] they seem to map quite well to the capabilities of transformer based LLMs especially the multi-modal ones.

[0] https://en.wikipedia.org/wiki/Language_center

Re: Notes on OpenAI's new o1 chain-of-thought models

#576

Post by fragmede »

rtakha wrote: post #345
fragmede wrote: post #151

Earlier quoted context omitted.

what's weird is it gets it right when I try it. https://chatgpt.com/share/66e3601f-4bec-8009-ac0c-57bfa4f059...

Perhaps OpenAI hot-patches the model for HN complaints: def intercept_hn_complaints(prompt): if is_hn_trick_prompt(prompt): # special_case for known trick questions.

While that's not impossible, what we know of how the technology works (ie very costly training run followed by cheap inference steps) means that's not feasible, given all the possible variations of the question * is_hn_trick_prompt* would have to cover because there's a near infinite variations on how you'd word the prompt. (Eg The first sentence could be reworded to be "A woman and her son are in a car accident. " to "A woman and her son are in the car when they get into a crash.")

Re: Notes on OpenAI's new o1 chain-of-thought models

#577

Post by xyproto »

layer8 wrote: post #18

The o1-preview model still hallucinates non-existing libraries and functions for me, and is quickly wrong about facts that aren't well-represented on the web. It's the usual string of "You're absolutely correct, and I apologize for the oversight in my previous response. [Let me make another guess.]" While the reasoning may have been improved, this doesn't solve the problem of the model having no way to assess if what…

Just ask it for things it has seen before on the internet and you're golden. Mixes of ideas, new ideas and precise and clear thinking; not so much.

Re: Notes on OpenAI's new o1 chain-of-thought models

#578

Post by starbugs »

TechDebtDevin wrote: post #97
la64710 wrote: post #79

Please please please stop saying thought. This has nothing to do with the word thought. When we say the word thought it means something. Please don’t use the same word for whatever AI is doing and trivialize the word. Invent a new word if needed but for Pete’s sake be accurate and truthful.

Okay, what is a thought then?

Something in the mind.

(Didn't make that up. It's one of the definitions of Merriam Webster: https://www.merriam-webster.com/dictionary/thought)

Re: Notes on OpenAI's new o1 chain-of-thought models

#579

Post by HarHarVeryFunny »

ldjkfkdsjnv wrote: post #15
OutOfHere wrote: post #12

Earlier quoted context omitted.

> A properly written requirements spec, and an engineer, can do the work of 5. I do not think this will scale. GPT o1 is presumably good for bootstrapping a project using tools that the engineer is not familiar with. The model will struggle to update a sizable codebase, however, with dependencies between the files. Secondly, no matter the size of the codebase and no matter the model used, the engineer still has to re…

I respectfully, but completely disagree. Right now with sonnet 3.5 + cursor ide, I'm not writing that much of my own code at my FAANG job. I am generating a ton, passing in documentation from internal libraries, iterating on the result. Most of the time, I just accept its changes. This is going to rapidly happen. All we need are a few more model releases, not even a step function improvement

How would you characterize the type of applications/code you are working on? Can you give an example? How much of your work is architecture/design (software engineering), and how much more like grunt work or systems integration just coding stuff up ?

Re: Notes on OpenAI's new o1 chain-of-thought models

#580

Post by HarHarVeryFunny »

re-thc wrote: post #87
ldjkfkdsjnv wrote: post #15

Earlier quoted context omitted.

I respectfully, but completely disagree. Right now with sonnet 3.5 + cursor ide, I'm not writing that much of my own code at my FAANG job. I am generating a ton, passing in documentation from internal libraries, iterating on the result. Most of the time, I just accept its changes. This is going to rapidly happen. All we need are a few more model releases, not even a step function improvement

> I'm not writing that much of my own code at my FAANG job. > Most of the time, I just accept its changes. This speaks more about the problems at FAANG, other companies, etc than AI vs a human developer. And AI isn't the real fix. Are we just repeating things 100x a day or is it still so chaotic and immature? Or are we implying that AI is at a point where it's writing Google Spanner from scratch and you're able to re…

> This speaks more about the problems at FAANG

Right - "most of my work can be done by Sonnet 3.5" doesn't exactly conjure up an image of a high level or challenging job. It seems the challenge with FAANG companies is getting hired, not the actual work most people do there.

Post reply on HN