Hi HN,
I’m considering getting an R9700, budget only for one unit currently. Is there anything useful I can do with local LLMs within its 32G VRAM? I’m also thinking to try and learn building a simple inference engine. Any thoughts?
Ask HN: What can I do with a single R9700?
1–8 of 8 posts
Re: Ask HN: What can I do with a single R9700?
#2For learning it's all you need. Pick a good small model, I would recommend Qwen3.5 there are models ranging from 0.8b params (will run on a smart phone) all the way up to 397b params .. I run the 122b param model daily with 128GB vram and get 20+tk/sec. regardless of the model, you can learn all about how to host and harness a model with any size
Re: Ask HN: What can I do with a single R9700?
#3For learning it's all you need. Pick a good small model, I would recommend Qwen3.5 there are models ranging from 0.8b params (will run on a smart phone) all the way up to 397b params .. I run the 122b param model daily with 128GB vram and get 20+tk/sec. regardless of the model, you can learn all about how to host and harness a model with any size
With this amount of VRAM, do you happen to be using a Strix Halo or a Mac?
Re: Ask HN: What can I do with a single R9700?
#4Plenty - you could do both with an R9700.
Unless it's CUDA only, a fair amount of software now supports AMD ROCm. And if you're already wanting to write an inference engine, 32G will be more than fine.
I've had a R9700 and it'd run the lower quants (4-5 bit?) of Qwen 3.5 27B reasonably attached to Hermes Agent. Not quite a 3090, but still respectable speeds.
Re: Ask HN: What can I do with a single R9700?
#5[flagged]
Re: Ask HN: What can I do with a single R9700?
#6Plenty - you could do both with an R9700. Unless it's CUDA only, a fair amount of software now supports AMD ROCm. And if you're already wanting to write an inference engine, 32G will be more than fine. I've had a R9700 and it'd run the lower quants (4-5 bit?) of Qwen 3.5 27B reasonably attached to Hermes Agent. Not quite a 3090, but still respectable speeds.
You say that as it's in the past, have you moved on to something else?
Re: Ask HN: What can I do with a single R9700?
#7[flagged]
Re: Ask HN: What can I do with a single R9700?
#8Plenty - you could do both with an R9700. Unless it's CUDA only, a fair amount of software now supports AMD ROCm. And if you're already wanting to write an inference engine, 32G will be more than fine. I've had a R9700 and it'd run the lower quants (4-5 bit?) of Qwen 3.5 27B reasonably attached to Hermes Agent. Not quite a 3090, but still respectable speeds.
[dead]