[deleted]
Learning to Reason with LLMs
11–20 of 1001 posts
Re: Learning to Reason with LLMs
#12What's with this how many r's in a strawberry thing I keep seeing?
This became something of a meme.
https://community.openai.com/t/incorrect-count-of-r-characte...
Re: Learning to Reason with LLMs
#13after weighing multiple factors including user experience, competitive advantage, and the option to pursue the chain of thought monitoring, we have decided not to show the raw chains of thought to users.
Re: Learning to Reason with LLMs
#14No direct indication of what “maximum test time” means, but if I’m reading the obscured language properly, the best scores on standardized tests were generated across a thousand samples with supplemental help provided.
Obviously, I hope everyone takes what any company says about the capabilities of its own software with a huge grain of salt. But it seems particularly called for here.
Re: Learning to Reason with LLMs
#15The model performance is driven by chain of thought, but they will not be providing chain of thought responses to the user for various reasons including competitive advantage. After the release of GPT4 it became very common to fine-tune non-OpenAI models on GPT4 output. I’d say OpenAI is rightly concerned that fine-tuning on chain of thought responses from this model would allow for quicker reproduction of their resu…
Can you explain what you mean by this?
Re: Learning to Reason with LLMs
#16Honestly, it doesn't matter for the end user if there are more tokens generated between the AI reply and human message. This is like getting rid of AI wrappers for specific tasks. If the jump in accuracy is actual, then for all practical purposes, we have a sufficiently capable AI which has the potential to boost productivity at the largest scale in human history.
Re: Learning to Reason with LLMs
#17[flagged]
Re: Learning to Reason with LLMs
#18yeah this is kinda cool i guess but 808 elo is still pretty bad for a model that can supposedly code like a human, i mean 11th percentile is like barely scraping by, and what even is the point of simulating codeforces if youre just gonna make a model that can barely compete with a decent amateur, and btw what kind of contest allows 10 submissions, thats not how codeforces works, and what about the time limits and memory limits and all that jazz, did they even simulate those, and btw how did they even get the elo ratings, is it just some arbitrary number they pulled out of their butt, and what about the model that got 1807 elo, is that even a real model or just some cherry picked result, and btw what does it even mean to "perform better than 93% of competitors" when the competition is a bunch of humans who are all over the place in terms of skill, like what even is the baseline for comparison
edit: i got confused with the Codeforce. it is indeed zero shot and O1 is potentially something very new I hope Anthropic and others will follow suit
any type of reasoning capability i'll take it !
Re: Learning to Reason with LLMs
#19oh wow, something you can roughly model as a diy in a base model. so impressive. yawn.
at least NVDA should benefit. i guess.
Re: Learning to Reason with LLMs
#20Congrats to OpenAI for yet another product that has nothing to do with the word "open"