Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Notes on OpenAI's new o1 chain-of-thought models

simonwillison.net

Post reply on HN
Print view View original

561–570 of 659 posts

Re: Notes on OpenAI's new o1 chain-of-thought models

#561

Post by lupire »

multjoy wrote: post #282
ramraj07 wrote: post #112

Earlier quoted context omitted.

Many of my PhD and post doc colleagues who emigrated from Korea, China and India who didn’t have English as the medium of instruction would struggle with this question. They only recover when you give them a hint. They’re some of the smartest people in general. If you try to stop stumping these models with trick questions and ask it straightforward reasoning systems it is extremely performant (O1 is definitely a step…

“Don’t be mean to LLMs, it isn’t their fault that they’re not actually intelligent”

"Don't anthropomorphize LLMs. They're hallucinating when they say they love that."

Re: Notes on OpenAI's new o1 chain-of-thought models

#562

Post by janalsncm »

ChicagoDave wrote: post #201
janalsncm wrote: post #157

Earlier quoted context omitted.

> We’re no closer to “AI” today than we were 20 years ago. 20 years ago we had barely figured out how to create superhuman agents to play chess. We have now created a new algorithm to solve Go, which is a much harder game. We then created an algorithm (alpha zero) to teach itself to play any game, and which became the best chess player in the world in hours. We next created a superhuman poker agent. Poker is even mor…

You can point a tool at a solution and certainly get results. Doesn’t mean it’s intelligent.

Solving difficult cognitive tasks is exactly what most people would call “intelligent”.

Re: Notes on OpenAI's new o1 chain-of-thought models

#563

Post by raincole »

ramraj07 wrote: post #112
pedrosorio wrote: post #110

Earlier quoted context omitted.

> It knows english at or above a level equal to most fluent speakers, and it also can produce output that is not just a likely output, but is a logical output This is not an apt description of the system that insists the doctor is the mother of the boy involved in a car accident when elementary understanding of English and very little logic show that answer to be obviously wrong. https://x.com/colin_fraser/status/183…

Many of my PhD and post doc colleagues who emigrated from Korea, China and India who didn’t have English as the medium of instruction would struggle with this question. They only recover when you give them a hint. They’re some of the smartest people in general. If you try to stop stumping these models with trick questions and ask it straightforward reasoning systems it is extremely performant (O1 is definitely a step…

I think you have particularly dumb colleagues then. If you post this question to an average STEM PhD in China (not even from China. In China) they'll get it right.

This question is the "unmisleading" version of a very common misleading question about sexism. ChatGPT learned the original, misleading version too well that it can't answer the unmisleading version.

Humans who don't have the original version ingrained in their brains will answer it with ease. It's not even a tricky question to humans.

Re: Notes on OpenAI's new o1 chain-of-thought models

#564

Post by jasondigitized »

williamdclt wrote: post #164
MPSimmons wrote: post #106

Earlier quoted context omitted.

The failure is in how you're using it. I don't mean this as a personal attack, but more to shed light on what's happening. A lot of people use LLMs as a search engine. It makes sense - it's basically a lossy compressed database of everything its ever read, and it generates output that is statistically likely - varying degrees of likeliness depending on the temperature, as well as how many times the particular weights…

> Treat it as a naive but intelligent intern That’s the problem: it’s a _terrible_ intern. A good intern will ask clarifying questions, tell me “I don’t know” or “I’m not sure I did it right”. LLMs do none of that, they will take whatever you ask and give a reasonable-sounding output that might be anything between brilliant and nonsense. With an intern, I don’t need to measure how good my prompting is, we’ll usually…

Have you ever worked with an intern? They have personalities and expectations that need to be managed. They get sick. The get tired. They want to punch you if you treat them like a 24-7 bird dog. It's so much easier to not let perfect be the enemy of the good and just rapid fire ALL day at a LLM for any and everything I need help with. You can also just not use the LLM. Interns need to be 'fed' work or the ROI ends upside down. Is a LLM as good as a top tier intern. No, but with a LLM I can have 10 pretty good interns by opening 10 tabs.

Re: Notes on OpenAI's new o1 chain-of-thought models

#566

Post by ijk »

awb wrote: post #465
berniedurfee wrote: post #381

Earlier quoted context omitted.

I’m starting to think this is an unsolvable problem with LLMs. The very act of “reasoning” requires one to know that they don’t know something. LLMs are giant word Plinko machines. A million monkeys on a million typewriters. LLMs are not interns. LLMs are assumption machines. None of the million monkeys or the collective million monkeys are “reasoning” or are capable of knowing. LLMs are a neat parlor trick and are s…

It probably depends on your problem space. In creative writing, I wonder if its even perceptible if the LLM is creating content at the boundaries of its knowledge base. But for programming or other falsifiable (and rapidly changing) disciplines it is noticeable and a problem. Maybe some evaluation of the sample size would be helpful? If the LLM has less than X samples of an input word or phrase it could include a cau…

In creative writing the problem becomes things like word choice and implications that have unexpected deviations from its expectations.

It can get really obvious when it's repeatedly using clichés. Both in repeated phrases and in trying to give every story the same ending.

Re: Notes on OpenAI's new o1 chain-of-thought models

#567

Post by cjonas »

MSFT_Edging wrote: post #491
cjonas wrote: post #487

Earlier quoted context omitted.

The difference is a junior cost 30-100$/hr and will take 2 days to complete the task. The LLM will do it in 20 seconds and cost 3c

Thank god we can finally end the scourge of interns to give the shareholders a little extra value. Good thing none of us ever started out as an intern.

I never said any of this will be good for society... In fact, I'm confident the current trajectory is going to cause wealth inequality at an entirely new level.

Underestimating the impact these models can have is a risk I'm trying to expose...

Re: Notes on OpenAI's new o1 chain-of-thought models

#568

Post by famouswaffles »

berniedurfee wrote: post #381
williamdclt wrote: post #164

Earlier quoted context omitted.

> Treat it as a naive but intelligent intern That’s the problem: it’s a _terrible_ intern. A good intern will ask clarifying questions, tell me “I don’t know” or “I’m not sure I did it right”. LLMs do none of that, they will take whatever you ask and give a reasonable-sounding output that might be anything between brilliant and nonsense. With an intern, I don’t need to measure how good my prompting is, we’ll usually…

I’m starting to think this is an unsolvable problem with LLMs. The very act of “reasoning” requires one to know that they don’t know something. LLMs are giant word Plinko machines. A million monkeys on a million typewriters. LLMs are not interns. LLMs are assumption machines. None of the million monkeys or the collective million monkeys are “reasoning” or are capable of knowing. LLMs are a neat parlor trick and are s…

LLMs mostly know what they know. Of course, that doesn't mean they're going to tell you.

https://news.ycombinator.com/item?id=41504226

Re: Notes on OpenAI's new o1 chain-of-thought models

#569

Post by ants_everywhere »

ryoshu wrote: post #385
williamdclt wrote: post #164

Earlier quoted context omitted.

> Treat it as a naive but intelligent intern That’s the problem: it’s a _terrible_ intern. A good intern will ask clarifying questions, tell me “I don’t know” or “I’m not sure I did it right”. LLMs do none of that, they will take whatever you ask and give a reasonable-sounding output that might be anything between brilliant and nonsense. With an intern, I don’t need to measure how good my prompting is, we’ll usually…

1000% this. LLMs can't say "I don't know" because they don't actually think. I can coach a junior to get better. LLMs will just act like they know what they are doing and give the wrong results to people who aren't practitioners. Good on OAI calling their model Strawberry because of Internet trolls. Reactive vs proactive.

I ask ChatGPT whether it knows things all the time. But it's almost never answers no.

As an experiment I asked it if it knew how to solve an arbitrary PDE and it said yes.

I then asked it if it could solve an arbitrary quintic and it said no.

So I guess it can say it doesn't know if it can prove to itself it doesn't know.

Re: Notes on OpenAI's new o1 chain-of-thought models

#570

Post by throwuxiytayq »

resters wrote: post #565

I pasted in some markdown content with symbolic logic expressions and proof examples and the model said I was violating the terms of service.

Welcome to The Future, where computation has terms of service and each workload you submit for execution is automatically verified to be in compliance.
Post reply on HN