Excellent, look forward to giving this a go. I was looking at: https://arxiv.org/abs/2506.18254 but your approach is even more general.
I really like RLPR for when you have a known-good answer to compare to as well!
Show HN: RULER – Easily apply RL to any agent
11–14 of 14 posts
Re: Show HN: RULER – Easily apply RL to any agent
#12Might end up being some confusion with the RULER benchmark from NVIDIA given the (somewhat shared) domain: https://github.com/NVIDIA/RULER EDIT: by shared I only mean the adjacency to LLMs/AI/ML, RL is a pretty big differentiator though and project looks great
Dang, hadn't seen that. Namespace collision strikes again.
Re: Show HN: RULER – Easily apply RL to any agent
#13Might end up being some confusion with the RULER benchmark from NVIDIA given the (somewhat shared) domain: https://github.com/NVIDIA/RULER EDIT: by shared I only mean the adjacency to LLMs/AI/ML, RL is a pretty big differentiator though and project looks great
Dang, hadn't seen that. Namespace collision strikes again.
yeah unforutnately for you this is one of the well known long context benchmarks. too late tho, soldier on.
Re: Show HN: RULER – Easily apply RL to any agent
#14how does o3 on the customer support agent task so dreadfully underperform qwen?