71–80 of 131 posts
Earlier quoted context omitted.
A lot of the TDP is reserved for running the shader units at full-power. My RTX 3070 Ti only pulls ~110w of it's 320w running CUDA inference on Gemma 26b and E4B.
My 5090 runs at full TDP(pretty much exactly 575W) when running inference through LM Studio.
Earlier quoted context omitted.
On actual market it’s $1100 vs $3200 now, right? I actually got mine at $2200 at cost in the before days.
Current lowest price for a new card on Newegg: $949.99 vs $3,699.99.
Earlier quoted context omitted.
My 5090 runs at full TDP(pretty much exactly 575W) when running inference through LM Studio.
Cap the power to 400W you won’t see much impact
I cannot understand why would a tech reviewer do that.
For those that use Blender, in their section about Blender: > We hope that, in the future, there will be real options other than NVIDIA for GPU-based rendering, as it is an area where competition is nearly non-existent. And Checking opendata.blender.org, a NVIDIA GeForce RTX 4080 Laptop GPU scores 5301.8, while Intel Arc Pro B70 is still at 3824.64. So there is still a bit more to go before Intel GPUs perform close t…
This is because Blender is in fact using CUDA?
It lacked software support the for the primary target application, running LLM. The officially supported vllm fork is 6 version behind mainline. It did not run the latest hot new open models on huggingface. Parallel two of B70 reduce token rate, not improve it. So, the software behind B70 is basically so far behind.
Intel Arc B70 when released, can only produce 1/3 of the token of RTX PRO 4500. Well, it also cost 1/3 of RTX PRO 4500. It lacked software support the for the primary target application, running LLM. The officially supported vllm fork is 6 version behind mainline. It did not run the latest hot new open models on huggingface. Parallel two of B70 reduce token rate, not improve it. So, the software behind B70 is basical…
There's a tradeoff between dense models and MoEs on memory usage vs. compute for the same quality. For example, Qwen3.5 27B and Qwen3.5 122B A10B have similar average performance across benchmarks. The 122B is much faster to run than the 27B (generates more tokens at the same compute). The 27B, on the other hand, uses ~4x less VRAM at low context lengths (less difference at high context lengths). Right now, different…
I still not see the point running these models. I say they produce plausible garbage, nowhere near quality of frontier models (when they work). Why can't Intel look beyond this nonsense state of affair and build something with 1TB of RAM or more? What I am trying to say, I am yet to see anything competitive in the market. Cards very much stalled in sub 100GB region and best corporations can do is throw something to r…
Earlier quoted context omitted.
Just ran llama-bench at home with the similar priced AMD AI PRO R9700 32G. The phoronix numbers look extremely low? Probably I misunderstand their test bench. Anyway, here are some numbers. Maybe someone with access to a B70 can post a comparison. Tried to use the same model as the article: llama-bench -m gpt-oss-20b-Q8_0.gguf -ngl 999 -p 2048 -n 128 AMD R9700 pp2048=3867 tg128=175 And a bigger model, because testing…
As of b8966, it is still not great. | model | size | params | backend | ngl | test | t/s | | --------------------- | --------: | ------: | ------- | --: | -----: | -------------: | | gpt-oss 20B MXFP4 MoE | 11.27 GiB | 20.91 B | SYCL | 999 | pp2048 | 851.81 ± 6.50 | | gpt-oss 20B MXFP4 MoE | 11.27 GiB | 20.91 B | SYCL | 999 | tg128 | 42.05 ± 1.99 | | gpt-oss 20B MXFP4 MoE | 11.27 GiB | 20.91 B | Vulkan | 999 | pp2048…
In that particular model family, the choices are 20B and 120B, so 20B higher quant fits in VRAM, while you'd be settling for 120B at a lower quant. Is it that 20B MXFP4 is comparable in performance so no need for Q8?
Or is the insight simply that there are better models available now and the emphasis is on gpt-oss-20b, not Q8?