Absolute Zero: Reinforced Self-Play Reasoning with Zero Data
1–10 of 20 posts
Re: Absolute Zero: Reinforced Self-Play Reasoning with Zero Data
#2"Despite using zero human-curated data, AZR achieves state-of-the-art results on diverse coding and math reasoning benchmarks, even outperforming models trained on large in-domain datasets. This demonstrates the potential for sophisticated reasoning skills to emerge purely through self-play without domain-specific supervision."
Re: Absolute Zero: Reinforced Self-Play Reasoning with Zero Data
#3To be clear, this is not a model trained on zero data, this is a pretrained model (Qwen 2.5 trained on 18 trillion tokens) finetuned using self-generated data grounded by a Python interpreter
Re: Absolute Zero: Reinforced Self-Play Reasoning with Zero Data
#4Pretty sure OpenAI and/or DeepMind have already been doing something very similar for a while already, just without publishing it.
Re: Absolute Zero: Reinforced Self-Play Reasoning with Zero Data
#5Warning: abuse of this technique may cause the model to go blind.
Re: Absolute Zero: Reinforced Self-Play Reasoning with Zero Data
#6Pretty sure OpenAI and/or DeepMind have already been doing something very similar for a while already, just without publishing it.
Agreed, it's a pretty obvious solution to the problems once you are immersed in the problem space. I think it's much harder to setup an efficient training pipeline for this which does every single little detail in the pipeline correctly while being efficient.
Re: Absolute Zero: Reinforced Self-Play Reasoning with Zero Data
#7For everyone who says “modern incentives forbid publishing negative results,” let this stand as a counterexample!
Re: Absolute Zero: Reinforced Self-Play Reasoning with Zero Data
#8To be clear, this is not a model trained on zero data, this is a pretrained model (Qwen 2.5 trained on 18 trillion tokens) finetuned using self-generated data grounded by a Python interpreter
I think at this point the initial process of exposing the empty model to all the available domain data in bulk is no longer interesting to many people. It's an obvious first step so it's barely mentioned anymore. What's currently worked on is what you do afterwards to get a useful tool in the end.
Re: Absolute Zero: Reinforced Self-Play Reasoning with Zero Data
#9For everyone who says “modern incentives forbid publishing negative results,” let this stand as a counterexample!
Why do you think it's a negative result?
The table on page 9 shows great results.
Re: Absolute Zero: Reinforced Self-Play Reasoning with Zero Data
#10Related to this: has anyone seen a model respond with “oh wait I was wrong…” when you follow-up with a “can you explain why this answer is right?”
I still find that my uses of GPT and others still struggle with a sort of tunnel vision.