Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Learning to Reason with LLMs

openai.com

Post reply on HN
Print view View original

71–80 of 1001 posts

Re: Learning to Reason with LLMs

#71

Post by apsec112 »

rfw300 wrote: post #39

A lot of skepticism here, but these are astonishing results! People should realize we’re reaching the point where LLMs are surpassing humans in any task limited in scope enough to be a “benchmark”. And as anyone who’s spent time using Claude 3.5 Sonnet / GPT-4o can attest, these things really are useful and smart! (And, if these results hold up, O1 is much, much smarter.) This is a nerve-wracking time to be a knowled…

Even without AI, it's gotten ~10,000 times easier to write software than in the 1950s (eg. imagine trying to write PyTorch code by hand in IBM 650 assembly), but the demand for software engineering has only increased, because demand increases even faster than supply does. Jevons paradox:

https://en.wikipedia.org/wiki/Jevons_paradox

Re: Learning to Reason with LLMs

#73

Post by afavour »

rfw300 wrote: post #39

A lot of skepticism here, but these are astonishing results! People should realize we’re reaching the point where LLMs are surpassing humans in any task limited in scope enough to be a “benchmark”. And as anyone who’s spent time using Claude 3.5 Sonnet / GPT-4o can attest, these things really are useful and smart! (And, if these results hold up, O1 is much, much smarter.) This is a nerve-wracking time to be a knowled…

> People should realize we’re reaching the point where LLMs are surpassing humans in any task limited in scope enough to be a “benchmark”.

Can you explain what this statement means? It sounds like you're saying LLMs are now smart enough to be able to jump through arbitrary hoops but are not able to do so when taken outside of that comfort zone. If my reading is correct then it sounds like skepticism is still warranted? I'm not trying to be an asshole here, it's just that my #1 problem with anything AI is being able to separate fact from hype.

Re: Learning to Reason with LLMs

#74

Post by minimaxir »

thelastparadise wrote: post #55

Wouldn't this introduce new economics into the LLM market? I.e. if the "thinking loop" budget is parameterized, users might pay more (much more) to spend more compute on a particular question/prompt.

Depends on how OpenAI prices it.

Given the need for chain-of-thoughts, and that would be budgeted as output, the new model will not be cheap nor fast.

EDIT: Pricing is out and it is definitely not teneable unless you really really have a use case for it.

Re: Learning to Reason with LLMs

#75

Post by aktuel »

thelastparadise wrote: post #41
dinobones wrote: post #4

Generating more "think out loud" tokens and hiding them from the user... Idk if I'm "feeling the AGI" if I'm being honest. Also... telling that they choose to benchmark against CodeForces rather than SWE-bench.

Why not? Isn't that basically what humans do? Sit there and think for a while before answering, going down different branches/chains of thought?

Sure, but if I want a human, I can hire a human. Humans also do many other things I don't want my LLM to do.

Re: Learning to Reason with LLMs

#76

Post by hi »

p1esk wrote: post #58

Do people see the new models in the web interface? Mine still shows the old models (I'm a paid subscriber).

> "o1 models are currently in beta - The o1 models are currently in beta with limited features. Access is limited to developers in tier 5 (check your usage tier here), with low rate limits (20 RPM). We are working on adding more features, increasing rate limits, and expanding access to more developers in the coming weeks!"

https://platform.openai.com/docs/guides/rate-limits/usage-ti...

Re: Learning to Reason with LLMs

#78

Post by rfw300 »

bigstrat2003 wrote: post #45
rfw300 wrote: post #39

A lot of skepticism here, but these are astonishing results! People should realize we’re reaching the point where LLMs are surpassing humans in any task limited in scope enough to be a “benchmark”. And as anyone who’s spent time using Claude 3.5 Sonnet / GPT-4o can attest, these things really are useful and smart! (And, if these results hold up, O1 is much, much smarter.) This is a nerve-wracking time to be a knowled…

I cannot, in fact, attest that they are useful and smart. LLMs remain a fun toy for me, not something that actually produces useful results.

It’s definitely the case that there are some programming workflows where LLMs aren’t useful. But I can say with certainty that there are many where they have become incredibly useful recently. The difference between even GPT-4 last year and C3.5/GPT-4o this year is profound.

I recently wrote a complex web frontend for a tool I’ve been building with Cursor/Claude and I wrote maybe 10% of the code; the rest with broad instructions. Had I done it all myself (or even with GitHub Copilot only) it would have taken 5 times longer. You can say this isn’t the most complex task on the planet, but it’s real work, and it matters a lot! So for increasingly many, regardless of your personal experience, these things have gone far beyond “useful toy”.

Re: Learning to Reason with LLMs

#80

Post by p1esk »

hobofan wrote: post #40

That naming scheme... Will the next model be named "1k", so that the subsequent models will be named "4o1k", and we can all go into retirement?

More like you will need to dip into your 401k fund early to pay for it after they raise the prices.
Post reply on HN