151–160 of 1001 posts
However, it does open up an interesting avenue for the future. Could you prompt-cache just the chain-of-thought reasoning bits?
Sounds great, but so does their "new flagship model that can reason across audio, vision, and text in real time" announced in May. [0] [0] https://openai.com/index/hello-gpt-4o/
Do people see the new models in the web interface? Mine still shows the old models (I'm a paid subscriber).
> "o1 models are currently in beta - The o1 models are currently in beta with limited features. Access is limited to developers in tier 5 (check your usage tier here), with low rate limits (20 RPM). We are working on adding more features, increasing rate limits, and expanding access to more developers in the coming weeks!" https://platform.openai.com/docs/guides/rate-limits/usage-ti...
Sounds great, but so does their "new flagship model that can reason across audio, vision, and text in real time" announced in May. [0] [0] https://openai.com/index/hello-gpt-4o/
Earlier quoted context omitted.
> Hopefully the open source community will figure out how to effectively match and replicate what OpenAI is doing. No need for that, there is a Reflection 70B model that does the exact same thing - with chains of thought being separated from the "final answer" via custom 'tag' tokens.
Wasn't this the model that was proven to have been faking their benchmarks recently? Or am I thinking of a different model?
The model performance is driven by chain of thought, but they will not be providing chain of thought responses to the user for various reasons including competitive advantage. After the release of GPT4 it became very common to fine-tune non-OpenAI models on GPT4 output. I’d say OpenAI is rightly concerned that fine-tuning on chain of thought responses from this model would allow for quicker reproduction of their resu…
Well played
> Unless otherwise specified, we evaluated o1 on the maximal test-time compute setting. Maximal test time is the maximum amount of time spent doing the “Chain of Thought” “reasoning”. So that’s what these results are based on. The caveat is that in the graphs they show that for each increase in test-time performance, the (wall) time / compute goes up exponentially . So there is a potentially interesting play here. Th…
I interpreted it to suggest that the product might include a user-facing “maximum test time” knob. Generating problem sets for kids? You might only need or want a basic level of introspection, even though you like the flavor of this model’s personality over that of its predecessors. Problem worth thinking long, hard, and expensively about? Turn that knob up to 11, and you’ll get a better-quality answer with no human-…
It's nice that people have taken the obvious extra-tokens/internal thoughts approach to a point where it actually works.
If this works, then automated programming etc., are going to actually be tractable. It's another world.
> We have found that the performance of o1 consistently improves with more reinforcement learning (train-time compute) and with more time spent thinking (test-time compute). Wow. So we can expect scaling to continue after all. Hyperscalers feeling pretty good about their big bets right now. Jensen is smiling. This is the most important thing. Performance today matters less than the scaling laws. I think everyone has…