Viewing profile — guilamu

Where does this "10x" comes from?

FTFY: Starting July 20 Claude Fable 5 will ve excluded from Pro plans.

https://phosh.mobi/faq/#so-what-phones-are-supported

Wouldn't you say that Valve is an exception to that rule?

Any source to backup this claim, pretty please?

If you're concerned about that do not give internet to your tv and use any kind of tv box instead (shield tv, apple tv, etc).

It's not. France: €0.149/kWh (~$0.175) US: ~$0.12–$0.14/kWh https://www.globalpetrolprices.com/France/electricity_prices...

You're right, I've certainly been a bit presumptuous to call this'a benchmark'. It is indeed a flawed test. Yet,It's been giving me the occasion to try some open source models and …

Most people, including me, beg to disagree. Better Call Saul was a masterpiece. https://www.metacritic.com/tv/better-call-saul/

Yeah as I said this a benchmark for my usecase only, a single use case, which is obvisouly not representative of everybody's needs. What strike me as very strange though is that 0 …

https://openrouter.ai/openai/gpt-5.5-pro 30/180 usd on Openrouter. Did I miss something?

When nothing is noted it's max reasoning (xhigh in copilot chat in vscode if available). The models not availble on copilot were tested through opencode (max reasoning) and deepsee…

Yes those two models were tested on my own PC (local inference using my own CPU/GPU). So something my be bugged on my setup. gemma4-26b should be far better than gemma4-e4b.

Yes, the prompt is slim by design. I might be wrong, but the point was to see what the model can do "on it's own". The eval prompt is quite extensive: https://github.com/guilamu/ll…

Haha, just fixed the date! I haven't evaluated the judge benchmark. You have everything needed in the repo to do so though, so be my guest. It took me a bit of time to put all this…

Yes Opus 4.7 fast (no reasoning) did a worst job than Sonnet 4.6 high (with reasoning) according to Gemini 3.1 Pro evaluation.

Just tested it on my homemade Wordpress+GravityForms benchmark and it's one of the worst model of the leaderboard performance wise and the worst value wise: https://github.com/guil…

Good point! I think I won't change anything right now or I'll have to remake all tests... I'll use your input for the Level 2 task I plan on working on.

Recently, GitHub Copilot silently dropped support for Claude Opus on Pro accounts. Since Opus was my go-to model for my daily workflow (developing WordPress plugins), I needed a re…

Indeed. I'm still sad thought :(