Viewing profile — meander_water

This is what I was looking for, thanks!

Sure, but then you would just use an image generation or multimodal model to generate that image. I don't think you'd want a weird looking svg.

Can someone explain what the pelican on a bicycle tests exactly? And why is it so important? I've never understood how it could translate to a useful task in real life.

Firstly, I don't have many issues with benchmarks per se. But I do have issues with leaderboards. And the AA index is touted by lots of people to argue that X model is better than …

The funny thing is that these leaderboards have become completely meaningless for end-users to make decisions on when to use what model. A single metric ranking is useless because …

1. The benchmark is run with a python script - https://github.com/sunblaze-ucb/exploitgym using an agent harness. I suspect they used codex. So the model has access to the environm…

Seems similar to Openrouter Fusion - https://openrouter.ai/docs/guides/routing/routers/fusion-rou...

I thought all model providers are doing this under the hood anyway in their UI? They certainly seem to when A/B testing different models, and Fable routes to Opus 4.8 when guardrai…

GPTZero is much better at handling humanized outputs. Also has a similar false positive rate to Pangram.

> However, it’s your job to go down the rabbit hole, learn the 100%, and sprinkle in your 3%. I would say that there is a big difference between stealing without acknowledgement, a…

> I don't think you should waste time reviewing every single line of code in here and just use AI to review it! > What you bring is the knowledge that the author nor the LLM doesn'…

Thanks, I didn't mean to be brusque, but I have seen a lot of these vibe tests lately that come to grand conclusions like "X model is better than Y" from the result of a single pro…

> So we ran it head-to-head against Claude Opus 4.8: same one-shot prompt, build a 3D platformer in raw WebGL from scratch Running a single one-shot prompt is not a benchmark, not …

Really curious to understand why I'm being downvoted. I don't think it's a particularly spicy take - Just choose the right tool for the job.

As someone who has built both react based frontends and html based ones (with htmx), there is a law of diminishing returns at play. To start off, writing a basic crud website with …

All the model releases we've seen this year have only made incremental improvements in benchmarks. This feels like the first release that feels like a significant step up in terms …

Not the first study, and they all largely report the same results: https://www.nature.com/articles/s41562-025-02259-6 https://www.theguardian.com/money/2019/feb/19/four-day-week-..…