101–110 of 124 posts
Why buy an overpriced Nvidia 4090 when you can get an AMD Halo Strix or Apple M3 Studio APU with 512GB or 128GB of Ram?
Nvidia has kept prices high and performance low for as long as it can and finally competition is here.
Even Intel can make APUs with tons of RAM.
Nvidia hopefully is squirming.
I'm sure this question has been asked before, but why not launch a GPU with more but slower ram? That would fit bigger models while still affordable...
More than twice as fast as Nvidia 4090 for AI.
Launched last week.
As other commenters have mentioned, the performance of this set up is probably not really great since there's not enough VRAM and lots of bits have to be moved between CPU and GPU RAM. That said, there are sub-256GB quants of DeepSeek-R1 out there (not the distilled versions). See https://unsloth.ai/blog/deepseekr1-dynamic I can't quantify the difference between these and the full FP8 versions of DSR1, but I've been…
Earlier quoted context omitted.
This article keeps getting posted but it runs a thinking model at 3-4 tokens/s. You might as well take a vacation if you ask it a question. It’s a gimmick and not a real solution.
If you value local compute and don't need massive speed, that's still twice as fast as most people can type.
I'm sure this question has been asked before, but why not launch a GPU with more but slower ram? That would fit bigger models while still affordable...
Did you miss the news about AMD Halo Strix? More than twice as fast as Nvidia 4090 for AI. Launched last week.
Earlier quoted context omitted.
What a teaser article! All this info for setting up the system, but no performance numbers.
That's because the OP is linking to the quickstart guide. There are benchmark numbers on the github's root page, but it does not appear to include the new deepseek yet: https://github.com/intel/ipex-llm/tree/main?tab=readme-ov-fi...
I'm sure this question has been asked before, but why not launch a GPU with more but slower ram? That would fit bigger models while still affordable...
Did you miss the news about AMD Halo Strix? More than twice as fast as Nvidia 4090 for AI. Launched last week.
Not in memory bandwidth which is all that matter for LLM inference.