Learning to Reason with LLMs • Hacker News Board

Re: Learning to Reason with LLMs

#241

Post by seydor » Thu, Sep 12, 2024, 6:11 PM UTC

wmf wrote: post #184

ARandumGuy wrote: post #137

One thing that makes me skeptical is the lack of specific labels on the first two accuracy graphs. They just say it's a "log scale", without giving even a ballpark on the amount of time it took. Did the 80% accuracy test results take 10 seconds of compute? 10 minutes? 10 hours? 10 days? It's impossible to say with the data they've given us. The coding section indicates "ten hours to solve six challenging algorithmic…

People have been celebrating the fact that tokens got 100x cheaper and now here's a new system that will use 100x more tokens.

If it 's reasoning correctly, it shouldnt need a lot of tokens because you don't need to correct it.

You only need to ask it to solve nuclear fusion once.

Re: Learning to Reason with LLMs

#242

Post by zamalek » Thu, Sep 12, 2024, 6:11 PM UTC

seydor wrote: post #93

valine wrote: post #8

The model performance is driven by chain of thought, but they will not be providing chain of thought responses to the user for various reasons including competitive advantage. After the release of GPT4 it became very common to fine-tune non-OpenAI models on GPT4 output. I’d say OpenAI is rightly concerned that fine-tuning on chain of thought responses from this model would allow for quicker reproduction of their resu…

The open source/weights models so far have proved that openAI doesn't have some special magic sauce. I m confident we ll soon have a model from Meta or others that s close to this level of reasoning. [Also consider that some of their top researchers have departed] On a cursory look, it looks like the chain of thought is a long series of chains of thought balanced on each step, with a small backtracking added whenever…

I suspect that the largest limiting factor for a competing model will be the dataset. Unless they somehow used GPT4 to generate the dataset somehow, this is an extremely novel dataset to have to build.

Re: Learning to Reason with LLMs

#243

Post by evrydayhustling » Thu, Sep 12, 2024, 6:11 PM UTC

changoplatanero wrote: post #227

evrydayhustling wrote: post #219

Just did some preliminary testing on decrypting some ROT cyphertext which would have been viable for a human on paper. The output was pretty disappointing: lots of "workish" steps creating letter counts, identifying common words, etc, but many steps were incorrect or not followed up on. In the end, it claimed to check its work and deliver an incorrect solution that did not satisfy the previous steps. I'm not one to j…

Hmm, are you sure it was using the o1 model and not gpt4o? I've been using the o1 model and it does consistently well at solving rotation ciphers.

o1-preview . Were you using common plaintexts by chance (e.g. proverbs) or ROT13 specifically? Mine use all the right steps but just can't string them together.

Re: Learning to Reason with LLMs

#244

Post by mewpmewp2 » Thu, Sep 12, 2024, 6:12 PM UTC

changoplatanero wrote: post #227

evrydayhustling wrote: post #219

Just did some preliminary testing on decrypting some ROT cyphertext which would have been viable for a human on paper. The output was pretty disappointing: lots of "workish" steps creating letter counts, identifying common words, etc, but many steps were incorrect or not followed up on. In the end, it claimed to check its work and deliver an incorrect solution that did not satisfy the previous steps. I'm not one to j…

Hmm, are you sure it was using the o1 model and not gpt4o? I've been using the o1 model and it does consistently well at solving rotation ciphers.

Does it do better than Claude, because Claude (3.5 sonnet) handled ROTs perfectly and was able to also respond in ROT.

Re: Learning to Reason with LLMs

#245

Post by ARandumGuy » Thu, Sep 12, 2024, 6:12 PM UTC

packetlost wrote: post #191

ARandumGuy wrote: post #137

One thing that makes me skeptical is the lack of specific labels on the first two accuracy graphs. They just say it's a "log scale", without giving even a ballpark on the amount of time it took. Did the 80% accuracy test results take 10 seconds of compute? 10 minutes? 10 hours? 10 days? It's impossible to say with the data they've given us. The coding section indicates "ten hours to solve six challenging algorithmic…

When one axis is on log scale and the other is linear with the plot points appearing linear-ish, doesn't it mean there's a roughly exponential relationship between the two axis?

It'd be more accurate to call it a logarithmic relationship, since compute time is our input variable. Which itself is a bit concerning, as that implies that modest gains in accuracy require exponentially more compute time.

In either case, that still doesn't excuse not labeling your axis. Taking 10 seconds vs 10 days to get 80% accuracy implies radically different things on how developed this technology is, and how viable it is for real world applications.

Which isn't to say a model that takes 10 days to get an 80% accurate result can't be useful. There are absolutely use cases where that could represent a significant improvement on what's currently available. But the fact that they're obfuscating this fairly basic statistic doesn't inspire confidence.

Re: Learning to Reason with LLMs

#246

Post by HarHarVeryFunny » Thu, Sep 12, 2024, 6:12 PM UTC

logicchains wrote: post #133

arresin wrote: post #119

> Unless otherwise specified, we evaluated o1 on the maximal test-time compute setting. Maximal test time is the maximum amount of time spent doing the “Chain of Thought” “reasoning”. So that’s what these results are based on. The caveat is that in the graphs they show that for each increase in test-time performance, the (wall) time / compute goes up exponentially . So there is a potentially interesting play here. Th…

Surprising that at run time it needs an exponential increase in thinking to achieved a linear increase in output quality. I suppose it's due to diminishing returns to adding more and more thought.

The exponential increase is presumably because of the branching factor of the tree of thoughts. Think of a binary tree who's number of leaf nodes doubles (= exponential growth) at each level.

It's not too surprising that the corresponding increase in quality is only linear - how much difference in quality would you expect between the best, say, 10 word answer to a question, and the best 11 word answer ?

It'll be interesting to see what they charge for this. An exponential increase in thinking time means an exponential increase in FLOPs/dollars.

Re: Learning to Reason with LLMs

#247

Post by trash_cat » Thu, Sep 12, 2024, 6:12 PM UTC

I think what it comes down to is accuracy vs speed. OpenAI clearly took steps here to improve the accuracy of the output which is critical in a lot of cases for application. Even if it will take longer, I think this is a good direction. I am a bit skeptical when it comes to the benchmarks - because they can be gamed and they don't always reflect real world scenarios. Let's see how it works when people get to apply it in real life workflows. One last thing, I wish they could elaborate more on >>"We have found that the performance of o1 consistently improves with more reinforcement learning (train-time compute) and with more time spent thinking (test-time compute)."<< Why don't you keep training it for years then to approach 100%? Am I missing something here?

Re: Learning to Reason with LLMs

#248

Post by eucalpytus » Thu, Sep 12, 2024, 6:12 PM UTC

airstrike wrote: post #188

This model is currently available for those accounts in Tier 5 and above, which requires "$1,000 paid [to date] and 30+ days since first successful payment" More info here: https://platform.openai.com/docs/guides/rate-limits/usage-ti...

I didn't know this founder's edition battle pass existed.

Re: Learning to Reason with LLMs

#249

Post by mewpmewp2 » Thu, Sep 12, 2024, 6:12 PM UTC

evrydayhustling wrote: post #243

changoplatanero wrote: post #227

Earlier quoted context omitted.

Hmm, are you sure it was using the o1 model and not gpt4o? I've been using the o1 model and it does consistently well at solving rotation ciphers.

o1-preview . Were you using common plaintexts by chance (e.g. proverbs) or ROT13 specifically? Mine use all the right steps but just can't string them together.

Could you share your prompt?

Re: Learning to Reason with LLMs

#250

Post by vjerancrnjak » Thu, Sep 12, 2024, 6:13 PM UTC

HPMOR wrote: post #120

A near perfect on AMC 12, 1900 CodeForces ELO, and silver medal IOI competitor. In two years, we'll have models that could easily win IMO and IOI. This is __incredible__!!

It depends on what they mean by "simulation". It sounds like o1 did not participate in new contests with new problems.

Any previous success of models with code generation focus was easily discovered to be a copy-paste of a solution in the dataset.

We could argue that there is an improvement in "understanding" if the code recall is vastly more efficient.