111–120 of 131 posts
this review was essentially pointless, they reviewed the card for a ton of workloads nobody in their right mind would pick it for, and left out the only use case where it makes sense. great job?
If you find a friendlier way to phrase it, you may find more people willing to discuss it.
Earlier quoted context omitted.
What makes you think Intel will leave the GPU space?
https://www.tomshardware.com/pc-components/gpus/intel-has-re... I've seen several stories like this. Which is a shame since Intel offers the best value GPUs on the market. I guess it's possible they'll still make workstation GPUs while skipping the consumer market.
Can we not have a PCIe card that's ASIC (and isn't GPU) with even DDR 4 or DDR 5 memory (Let's say 128 GB) onboard and being able to shove four of them on a consumer grade motherboard and then being utilized in parallel? Noob question.
Bandwidth on that memory interface and setup for dual channel would be significantly worse than Strix Halo, which already exists and could be an entire compute setup with no need for an ASIC.
this review was essentially pointless, they reviewed the card for a ton of workloads nobody in their right mind would pick it for, and left out the only use case where it makes sense. great job?
There's a tradeoff between dense models and MoEs on memory usage vs. compute for the same quality. For example, Qwen3.5 27B and Qwen3.5 122B A10B have similar average performance across benchmarks. The 122B is much faster to run than the 27B (generates more tokens at the same compute). The 27B, on the other hand, uses ~4x less VRAM at low context lengths (less difference at high context lengths). Right now, different…
DGX Spark is at the compute level of 5070. Its main issue is low memory bandwidth, i.e. it has quite fast token prefill but awful token generation. Strix Halo is just slow on every metric and used to be a cheap way to get 128GB unified RAM (now its prices are comparable to DGX Spark).
Something that is also cool with these cards is proper SR-IOV without hassle. Arc pro cards make for nice graphical acceleration devices for vms. I know ai gets all the hype but I also appreciate being able to accelerate multiple workstations with a single gpu and still get decent frametimes.
Does the B70 definitely support SR-IOV from day one?
Intel Arc B70 when released, can only produce 1/3 of the token of RTX PRO 4500. Well, it also cost 1/3 of RTX PRO 4500. It lacked software support the for the primary target application, running LLM. The officially supported vllm fork is 6 version behind mainline. It did not run the latest hot new open models on huggingface. Parallel two of B70 reduce token rate, not improve it. So, the software behind B70 is basical…
What you say is not consistent with TFA. The parent article shows that B70 is faster than RTX 4000. RTX 4500 is faster than RTX 4000, but it cannot be more than 3 times faster, not even more than 2 times faster. The parent article is consistent with RTX 4500 being faster than B70 for ML inference, but by a much smaller ratio, e.g. less than 50% faster. If you know otherwise, please point to the source. If you have ru…
Can we not have a PCIe card that's ASIC (and isn't GPU) with even DDR 4 or DDR 5 memory (Let's say 128 GB) onboard and being able to shove four of them on a consumer grade motherboard and then being utilized in parallel? Noob question.
Something that is also cool with these cards is proper SR-IOV without hassle. Arc pro cards make for nice graphical acceleration devices for vms. I know ai gets all the hype but I also appreciate being able to accelerate multiple workstations with a single gpu and still get decent frametimes.
I bought an Arc A770 expecting that this feature would materialise based on it being available on other contemporary Intel GPUs, but it never did. Does the B70 definitely support SR-IOV from day one?
Earlier quoted context omitted.
LLMs are memory bandwidth bound not compute bound.
LLMs are bound by both and depends on the hardware which factor is higher.
Even a 5090 has something like 50-60 flops per byte of bandwidth, you just can't saturate the compute without running large batches. (At least at inference, prefill is obviously more compute bound).