Hi HN,
I’m considering getting an R9700, budget only for one unit currently. Is there anything useful I can do with local LLMs within its 32G VRAM? I’m also thinking to try and learn building a simple inference engine. Any thoughts?
Ask HN: What can I do with a single R9700?
1–10 of 10 posts
Re: Ask HN: What can I do with a single R9700?
#2For learning it's all you need. Pick a good small model, I would recommend Qwen3.5 there are models ranging from 0.8b params (will run on a smart phone) all the way up to 397b params .. I run the 122b param model daily with 128GB vram and get 20+tk/sec. regardless of the model, you can learn all about how to host and harness a model with any size
Re: Ask HN: What can I do with a single R9700?
#3For learning it's all you need. Pick a good small model, I would recommend Qwen3.5 there are models ranging from 0.8b params (will run on a smart phone) all the way up to 397b params .. I run the 122b param model daily with 128GB vram and get 20+tk/sec. regardless of the model, you can learn all about how to host and harness a model with any size
With this amount of VRAM, do you happen to be using a Strix Halo or a Mac?
Re: Ask HN: What can I do with a single R9700?
#4Plenty - you could do both with an R9700.
Unless it's CUDA only, a fair amount of software now supports AMD ROCm. And if you're already wanting to write an inference engine, 32G will be more than fine.
I've had a R9700 and it'd run the lower quants (4-5 bit?) of Qwen 3.5 27B reasonably attached to Hermes Agent. Not quite a 3090, but still respectable speeds.
Re: Ask HN: What can I do with a single R9700?
#5[flagged]
Re: Ask HN: What can I do with a single R9700?
#6Plenty - you could do both with an R9700. Unless it's CUDA only, a fair amount of software now supports AMD ROCm. And if you're already wanting to write an inference engine, 32G will be more than fine. I've had a R9700 and it'd run the lower quants (4-5 bit?) of Qwen 3.5 27B reasonably attached to Hermes Agent. Not quite a 3090, but still respectable speeds.
You say that as it's in the past, have you moved on to something else?
Re: Ask HN: What can I do with a single R9700?
#7[flagged]
Re: Ask HN: What can I do with a single R9700?
#8Plenty - you could do both with an R9700. Unless it's CUDA only, a fair amount of software now supports AMD ROCm. And if you're already wanting to write an inference engine, 32G will be more than fine. I've had a R9700 and it'd run the lower quants (4-5 bit?) of Qwen 3.5 27B reasonably attached to Hermes Agent. Not quite a 3090, but still respectable speeds.
[dead]
Re: Ask HN: What can I do with a single R9700?
#9For learning it's all you need. Pick a good small model, I would recommend Qwen3.5 there are models ranging from 0.8b params (will run on a smart phone) all the way up to 397b params .. I run the 122b param model daily with 128GB vram and get 20+tk/sec. regardless of the model, you can learn all about how to host and harness a model with any size
With this amount of VRAM, do you happen to be using a Strix Halo or a Mac?
It's a Dell Pro Max GB10. I have two that I can bridge together to run larger models across the 256GB pool but I only bridge them when I am experimenting with larger models and they will run fine but at lower tk/s with really only marginal reasoning gains, so I tend to stick to medium models (122b) on one GB10 and I use the other to serve gen models for music, image and video gen.
Re: Ask HN: What can I do with a single R9700?
#10Plenty - you could do both with an R9700. Unless it's CUDA only, a fair amount of software now supports AMD ROCm. And if you're already wanting to write an inference engine, 32G will be more than fine. I've had a R9700 and it'd run the lower quants (4-5 bit?) of Qwen 3.5 27B reasonably attached to Hermes Agent. Not quite a 3090, but still respectable speeds.
You say that as it's in the past, have you moved on to something else?
No, the R9700 I have got fried by a power supply fault, and GPU replacements are not always trivial.
Of the two affordable options for good 32G compute, the R9700 would be the best of the two (compared to the Arc Pro B70).