21–30 of 273 posts
For reference, here is the terminal-bench leaderboard: https://www.tbench.ai/leaderboard Looks like it doesn't get close to GPT-5, Claude 4, or GLM-4.5, but still does reasonably well compared to other open weight models. Benchmarks are rarely the full story though, so time will tell how good it is in practice.
tbh companies like anthopic, openai, create custom agents for specific benchmarks
Earlier quoted context omitted.
Those Qwen3 2507 models are the local creme-de-la-creme right now. If you've got any sort of GPU and ~32gb of RAM to play with, the A3B one is great for pair-programming tasks.
Do you happen to know if it can be run via an eGPU enclosure with f.ex. RTX 5090 inside, under Linux? I'm considering buying a Linux workstation lately and I want it full AMD. But if I can just plug an NVIDIA card via an eGPU card for self-hosting LLMs then that would be amazing.
Earlier quoted context omitted.
Do you happen to know if it can be run via an eGPU enclosure with f.ex. RTX 5090 inside, under Linux? I'm considering buying a Linux workstation lately and I want it full AMD. But if I can just plug an NVIDIA card via an eGPU card for self-hosting LLMs then that would be amazing.
I’m running Ollama on 2 eGPUs over Thunderbolt. Works well for me. You’re still dealing with an NVDIA device, of course. The connection type is not going to change that hassle.
Though I have to ask: why two eGPUs? Is the LLM software smart enough to be able to use any combination of GPUs you point it at?
It's a hybrid reasoning model. It's good with tool calls and doesn't think too much about everything, but it regularly uses outdated tool formats randomly instead of the standard JSON format. I guess the V3 training set has a lot of those.
What formats? I thought the very schema of json is what allows these LLMs to enforce structured outputs at the decoder level? I guess you can do it with any format, but why stray from json?
or this: ``` executeshell{"command": "pwd && ls -la"} ```
Prompting it to use the right format doesn't seem to work. Claude, Gemini, GPT5, and GLM 4.5, don't do that. To accomodate DeepSeek, the tiny agent that I'm building will have to support all the weird formats.
Earlier quoted context omitted.
tbh companies like anthopic, openai, create custom agents for specific benchmarks
Do you have a source for this? I’m intrigued
Earlier quoted context omitted.
What formats? I thought the very schema of json is what allows these LLMs to enforce structured outputs at the decoder level? I guess you can do it with any format, but why stray from json?
Sometimes it will randomly generate something like this in the body of the text: ``` executeshell command echo "" >> novels/AI_Voodoo_Romance/chapter-1-a-new-dawn.txt ``` or this: ``` executeshell {"command": "pwd && ls -la"} ``` Prompting it to use the right format doesn't seem to work. Claude, Gemini, GPT5, and GLM 4.5, don't do that. To accomodate DeepSeek, the tiny agent that I'm building will have to support all…
Earlier quoted context omitted.
Those Qwen3 2507 models are the local creme-de-la-creme right now. If you've got any sort of GPU and ~32gb of RAM to play with, the A3B one is great for pair-programming tasks.
I use it on a 24gb gpu Tesla P40. Very happy with the result.