Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Learning to Reason with LLMs

openai.com

Post reply on HN
Print view View original

81–90 of 1001 posts

Re: Learning to Reason with LLMs

#81

Post by deisteve »

qt31415926 wrote: post #56
deisteve wrote: post #18

yeah this is kinda cool i guess but 808 elo is still pretty bad for a model that can supposedly code like a human, i mean 11th percentile is like barely scraping by, and what even is the point of simulating codeforces if youre just gonna make a model that can barely compete with a decent amateur, and btw what kind of contest allows 10 submissions, thats not how codeforces works, and what about the time limits and mem…

808 ELO was for GPT-4o. I would suggest re-reading more carefully

you are right i read the charts wrong. O1 has significant lead over GPT-4o in the zero shot examples

honestly im spooked

Re: Learning to Reason with LLMs

#83

Post by yunohn »

The generated chain of thought for their example is incredibly long! The style is kind of similar to how a human might reason, but it's also redundant and messy at various points. I hope future models will be able to optimize this further, otherwise it'll lead to exponential increases in cost.

Re: Learning to Reason with LLMs

#84

Post by bongodongobob »

bigstrat2003 wrote: post #45
rfw300 wrote: post #39

A lot of skepticism here, but these are astonishing results! People should realize we’re reaching the point where LLMs are surpassing humans in any task limited in scope enough to be a “benchmark”. And as anyone who’s spent time using Claude 3.5 Sonnet / GPT-4o can attest, these things really are useful and smart! (And, if these results hold up, O1 is much, much smarter.) This is a nerve-wracking time to be a knowled…

I cannot, in fact, attest that they are useful and smart. LLMs remain a fun toy for me, not something that actually produces useful results.

At this point, you're either saying "I don't understand how to prompt them" or "I'm a Luddite". They are useful, here to stay, and only getting better.

Re: Learning to Reason with LLMs

#86

Post by patapong »

Very interesting. I guess this is the strawberry model that was rumoured.

I am a bit surprised that this does not beat GPT-4o for personal writing tasks. My expectations would be that a model that is better at one thing is better across the board. But I suppose writing is not a task that generally requires "reasoning steps", and may also be difficult to evaluate objectively.

Re: Learning to Reason with LLMs

#87

Post by Hansenq »

Reading through the Chain of Thought for the provided Cipher example (go to the example, click "Show Chain of Thought") is kind of crazy...it literally spells out every thinking step that someone would go through mentally in their head to figure out the cipher (even useless ones like "Hmm"!). It really seems like slowing down and writing down the logic it's using and reasoning over that makes it better at logic, similar to how you're taught to do so in school.

Re: Learning to Reason with LLMs

#88

Post by k2xl »

Pricing page updated for O1 API costs.

https://openai.com/api/pricing/

$15.00 / 1M input tokens $60.00 / 1M output tokens

For o1 preview

Approx 3x the price of gpt4o.

o1-mini $3.00 / 1M input tokens $12.00 / 1M output tokens

About 60% of the cost of gpt4o. Much more expensive than gpt4o-mini.

Curious on the performance/tokens per second for these new massive models.

Re: Learning to Reason with LLMs

#89

Post by XCSme »

modeless wrote: post #33

> We have found that the performance of o1 consistently improves with more reinforcement learning (train-time compute) and with more time spent thinking (test-time compute). Wow. So we can expect scaling to continue after all. Hyperscalers feeling pretty good about their big bets right now. Jensen is smiling. This is the most important thing. Performance today matters less than the scaling laws. I think everyone has…

Nvidia stock go brrr...

Re: Learning to Reason with LLMs

#90

Post by hi »

BUG: https://openai.com/index/reasoning-in-gpt/

> o1 models are currently in beta - The o1 models are currently in beta with limited features. Access is limited to developers in tier 5 (check your usage tier here), with low rate limits (20 RPM). We are working on adding more features, increasing rate limits, and expanding access to more developers in the coming weeks!

https://platform.openai.com/docs/guides/reasoning/reasoning

Post reply on HN