Viewing profile — Danau5tin

Great question. Today the answer is that there is nothing stopping this, but you are right this is a 100% valid reward hacking possibility. No signs of it in the run I did, but for…

Very good point. Sifting through noisy data and creating curated datasets could be of great value. Perhaps worth a project by itself!

Very cool! Thank you

Strangely, I did not. I was expecting it and looking out for it the whole time. At least I never say a warning!

Yes, thank you. Typo!

Yes exactly, and also there is auto-research by Karpathy, Fable-5 on parameter golf etc which point to the direction of highly-assisted / autonomous AI training as the future. Wild…

I am also curious! The inner-RL-loop models are only trained once, then disgarded. But the outer-RL-loop model is trained on the same tasks over and over again. I imagine it would …

Yes I do agree with this. I believe we are shifting from "make the model good" (prompt/context engineering, etc) to "define good for the model" (success criteria/rubrics). Over tim…

I chose the key technical decision and direction (such as the system architecture, the tasks to train on, the stack of Tinker, Prime-RL & Runpod - all of which I know well) etc. Th…

Yes I'd heard the labs (Anthropic mostly) speaking about LLMs training LLMs, so I wanted to make things a little more concrete and test it out myself! Essentially you are correct t…

Thank you!

Mainly Fable, but It was me who wanted to emojis added hah. I also of course edited the README by hand (crazy I know), but the code is entirely fable

I RL-trained an agent whose job is to write RL training jobs for smaller models, and open-sourced the whole thing. The setup is two nested RL loops: - Outer loop: the trainer agent…

My RL trained multi-agent-coding model Orca-Agent-v0.1-14B reached a 167% higher relative score than its base model on Stanford's TerminalBench. I've open sourced everything. *What…

This weekend I built a multi-agent coding system which, quite unexpectedly, beat Claude Code on Stanford's Terminal Bench! The architecture is straightforward, consisting of an orc…

Hitting a million brick walls with multi-turn RL training isn't fun, so I thought I would try something new to climb Stanford's leaderboard for now! So this weekend I was just tink…

Exactly my first thought when I realised the cost! Currently LoRA is not supported by rLLM (The team told me they aim to support in next release), but it is certainly possible to p…

After training calculator agent via RL, I really wanted to go bigger! So I built RL infrastructure for training long-horizon terminal/coding agents that scales from 2x A100s to 32x…