1–7 of 7 posts
Also, Claude Code tends to make very broad search requests, and I keep getting an error from MCP about exceeding 25,000 characters. It happens quite often.
What would you recommend?
Invest in a local inference server and run Qwen3. At this point it will still cost less than two pro accounts.
> What would you recommend? Invest in a local inference server and run Qwen3. At this point it will still cost less than two pro accounts.
> What would you recommend? Invest in a local inference server and run Qwen3. At this point it will still cost less than two pro accounts.
What hardware do you suggest? :)
Nvidia hardware is cheap as chips right now. If you got 2x 3060 12gb cards (or a 24gb 4090), you'd have 24gb of CUDA-accelerated VRAM to play with for inference and finetuning. It should be plenty to fit the smaller SOTA models like GLM-4.5 Air, Qwen3 30b A3B, and Llama Scout, and definitely enough to start layering the giant 100b+ parameter options.
That's what I'd get, at least.
Earlier quoted context omitted.
What hardware do you suggest? :)
Iunno, whatever you can afford? Nvidia hardware is cheap as chips right now. If you got 2x 3060 12gb cards (or a 24gb 4090), you'd have 24gb of CUDA-accelerated VRAM to play with for inference and finetuning. It should be plenty to fit the smaller SOTA models like GLM-4.5 Air, Qwen3 30b A3B, and Llama Scout, and definitely enough to start layering the giant 100b+ parameter options. That's what I'd get, at least.
Are they good enough compared to Sonnet 4?
I’ve also used Gemini 2.5 Pro and Flash, and they’re worse. But they’re much bigger, not just 30B.
Earlier quoted context omitted.
Iunno, whatever you can afford? Nvidia hardware is cheap as chips right now. If you got 2x 3060 12gb cards (or a 24gb 4090), you'd have 24gb of CUDA-accelerated VRAM to play with for inference and finetuning. It should be plenty to fit the smaller SOTA models like GLM-4.5 Air, Qwen3 30b A3B, and Llama Scout, and definitely enough to start layering the giant 100b+ parameter options. That's what I'd get, at least.
> GLM-4.5 Air, Qwen3 30b A3B, and Llama Scout Are they good enough compared to Sonnet 4? I’ve also used Gemini 2.5 Pro and Flash, and they’re worse. But they’re much bigger, not just 30B.
You might be able to try out Qwen3 via API to see if it suits your needs. Their 30b MOE is really impressive, and the 480b one can only be better (presumably).
> What would you recommend? Invest in a local inference server and run Qwen3. At this point it will still cost less than two pro accounts.