31–40 of 117 posts
I understand their reasoning but it’s still a shame Framework opted for a proprietary motherboard instead of mini-ITX with a socket CPU and GPU. I value repairability over a little bit of extra performance.
It’s fully possible that I’ve missed something, but it doesn’t appear to be proprietary to me.
I typed my (masked) email into the input, pressed tab and enter, it redirected me to some cloudflare captcha marketing site. I thought at first it didn't want the masked email, but then it worked when I clicked with my mouse. Apparently there are 3 invisible focus targets in between! Dear people who create websites, these things are important, they should work!
Here's me wanting to drop thousands on this so I can save a few bucks a day by not having to call cloud llms for simple distillation tasks. Lol.
I understand their reasoning but it’s still a shame Framework opted for a proprietary motherboard instead of mini-ITX with a socket CPU and GPU. I value repairability over a little bit of extra performance.
The whole point is that these APUs present a pretty unique value proposition (lowish performance GPUs with massive amounts of RAM attached), and framework is afaik the only vendor shipping them on standard mini-ITX boards
Im less interested in the memory than the memory bandwidth. The current system with 128GB can load pretty big models, but its meaningless unless you want to wait 40 minutes per prompt. I've had best results with Qwen3.6-35B-A3B, which uses 40GB of memory, but only uses 3 billion parameters per token which helps with throughput. Until memory bandwidth significantly improves I just can't see myself wanting to use all t…
I have an NVIDIA Spark thingy (the ASUS one) and it's the same problem there, though the prefill side is superior to the Ryzen ones. But I actually think 128GB is too little. There are some compelling models that are above what can fit in that at reasonable quants (e.g. DeepSeek V4 Flash) but could if the system was 256GB. If RAM prices weren't so f*cked I think we'd be seeing 256GB and even 512GB unified memory syst…
Im less interested in the memory than the memory bandwidth. The current system with 128GB can load pretty big models, but its meaningless unless you want to wait 40 minutes per prompt. I've had best results with Qwen3.6-35B-A3B, which uses 40GB of memory, but only uses 3 billion parameters per token which helps with throughput. Until memory bandwidth significantly improves I just can't see myself wanting to use all t…
If your performance is significantly slower then you are probably doing it in CPU - there was some fiddling required to get it to use GPU (I use llama.cpp)
Ryzen AI Max Plus Pro 495 How does a CEO read this and not immediately fire their entire marketing team?
I don't see anything wrong with the name.
Ryzen AI Max Plus Pro 495 How does a CEO read this and not immediately fire their entire marketing team?
AI means it has npu, Max+ is marking the memory channel, PRO is a normal label for chips that have extra security baked in, it has been this way since forever. I don't see anything wrong with the name.
Anyone who thinks they are going to serve some 100+ GB LLM locally, remember that memory bandwidth becomes a key limitation for large models. While you might be able to load a model, token generation can be very slow. MoE models like Qwen3.5-122B-A10B work decently fast, but dense models of a decent size are slow and you won’t want to use them.
I’ve got a 395 system, and found that I’m quite happy with Qwen3.6-35B-A3B, generating at around 50 t/s, but the dense 27B model is 20-25 t/s and that’s the lower limit I’m willing to tolerate. So a 70b dense model is just not going to happen. That means you can’t really use that much RAM.
A reasonable use case is to have multiple smaller models loaded- you can have an image generation model loaded along with the text model. Or you can use this computer for development simultaneously with serving LLMs. Those ideas work okay. But trying to load up a single giant model is going to test your patience.
> Seize the means of computation Feels like they blatantly ripped that off the title of a Cory Doctorow book [0]. [0]: https://en.wikipedia.org/wiki/The_Internet_Con