Viewing profile — AIReach

BenchmarkAggregator is an open-source framework for comprehensive LLM evaluation across cutting-edge benchmarks like GPQA Diamond, MMLU Pro, and Chatbot Arena. It offers unbiased c…

I've developed ModelClash, an open-source framework for LLM evaluation that could offer some potential advantages over static benchmarks: * Automatic challenge generation, reducing…

Hi! I've developed ModelClash, an open-source framework for LLM evaluation that could offer some potential advantages over static benchmarks: * Automatic challenge generation, redu…

The Long Multiplication Benchmark evaluates Large Language Models (LLMs) on their ability to handle and utilize long contexts to solve multiplication problems. Despite long multipl…

You've brought up good point. But let's consider this: Current LLMs are capable of generating code that rivals human output. However, two major challenges persist. First, there's n…

PullRequestBenchmark introduces a new standard for Large Language Models (LLMs)—reviewing pull requests with human-like discernment. Approaching full developer job automation, this…

Thanks and I agree. Especially because it's easily scalable to longer contexts. It's trivial to use the "framework to" run the tests but I cannot prioritize it right now so either …