Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Intel Arc Pro B70 Review

pugetsystems.com

Post reply on HN
Print view View original

51–60 of 131 posts

Re: Intel Arc Pro B70 Review

#51

Post by BoredPositron »

2001zhaozhao wrote: post #43

There's a tradeoff between dense models and MoEs on memory usage vs. compute for the same quality. For example, Qwen3.5 27B and Qwen3.5 122B A10B have similar average performance across benchmarks. The 122B is much faster to run than the 27B (generates more tokens at the same compute). The 27B, on the other hand, uses ~4x less VRAM at low context lengths (less difference at high context lengths). Right now, different…

I am working mostly with image models so we do a lot of fun times and the card fits perfectly here. Performance isn't great but it can just tug along in the background.àp

Re: Intel Arc Pro B70 Review

#53

Post by lukan »

marshray wrote: post #22
driverdan wrote: post #10

From what I've read the Intel drivers are terrible and holding back using them for LLMs.

I don't know about LLMs, but I tried an Intel card when Ubuntu Wayland couldn't initialize a 2 year old Nvidia. It just works.

That is just Linux and politics. Linux wants to force vendors to open source theirs, Intel plays along, Nvidia as the market lead does not, so you have to use their proprietary one, which most distros do not ship by default.

Re: Intel Arc Pro B70 Review

#54

Post by magicalhippo »

canpan wrote: post #31
SparkyMcUnicorn wrote: post #8

Here are some llama.cpp benchmarks for it: https://www.phoronix.com/review/intel-arc-pro-b70-linux/3

Just ran llama-bench at home with the similar priced AMD AI PRO R9700 32G. The phoronix numbers look extremely low? Probably I misunderstand their test bench. Anyway, here are some numbers. Maybe someone with access to a B70 can post a comparison. Tried to use the same model as the article: llama-bench -m gpt-oss-20b-Q8_0.gguf -ngl 999 -p 2048 -n 128 AMD R9700 pp2048=3867 tg128=175 And a bigger model, because testing…

For reference in case it's interesting to someone, a 5090 on Windows 11 with CUDA 13.1

  | model                 |       size |   params | backend  | ngl |   test |              t/s |
  | --------------------- | ---------: |--------: | -------- | --: |------: |----------------: |
  | gpt-oss 20B MXFP4 MoE |  11.27 GiB |  20.91 B | CUDA     | 999 | pp2048 | 10179.12 ± 52.86 |
  | gpt-oss 20B MXFP4 MoE |  11.27 GiB |  20.91 B | CUDA     | 999 |  tg128 |    326.82 ± 7.82 |
  | qwen35 27B Q6_K       |  23.87 GiB |  26.90 B | CUDA     | 999 | pp2048 |   3129.92 ± 5.12 |
  | qwen35 27B Q6_K       |  23.87 GiB |  26.90 B | CUDA     | 999 |  tg128 |     53.45 ± 0.15 |
  
  build: 9d34231bb (8929)

  gpt-oss-20b-MXFP4.gguf
  Qwen3.6-27B-UD-Q6_K_XL.gguf
Using MXFP4 of GPT-OSS because it was trained quantization-aware for this quantization type, and it's native to the 50xx.

Re: Intel Arc Pro B70 Review

#55

Post by jonhohle »

MostlyStable wrote: post #6

Is Intel still making GPUs? I have heard so many conflicting things about will they/won't they stay in the market.

I thought I had read that too and went to look for clarification and found that they’re just moving to a single architecture for their cards. Seemed reasonable.

Re: Intel Arc Pro B70 Review

#56

Post by rao-v »

jbellis wrote: post #52

How should I update my simplistic understanding that decode is bw-bound with these results that show the B70 decoding faster than a 4090 (about 50% more bw)?

I doubt you'd get the same sort of result on a modern-ish MOE or dense model via a more standard inference engine like llama.cpp or VLLM. I don't think MLPerf is a reasonable benchmark at this point.

Edit: Here is a simple llama.cpp compare where the token gen results match the rule of thumb.

https://www.reddit.com/r/LocalLLaMA/comments/1st6lp6/nvidia_...

Re: Intel Arc Pro B70 Review

#57

Post by storus »

2001zhaozhao wrote: post #43

There's a tradeoff between dense models and MoEs on memory usage vs. compute for the same quality. For example, Qwen3.5 27B and Qwen3.5 122B A10B have similar average performance across benchmarks. The 122B is much faster to run than the 27B (generates more tokens at the same compute). The 27B, on the other hand, uses ~4x less VRAM at low context lengths (less difference at high context lengths). Right now, different…

DGX Spark is at the compute level of 5070. Its main issue is low memory bandwidth, i.e. it has quite fast token prefill but awful token generation. Strix Halo is just slow on every metric and used to be a cheap way to get 128GB unified RAM (now its prices are comparable to DGX Spark).

Re: Intel Arc Pro B70 Review

#58

Post by luckydata »

this review was essentially pointless, they reviewed the card for a ton of workloads nobody in their right mind would pick it for, and left out the only use case where it makes sense. great job?

Re: Intel Arc Pro B70 Review

#59

Post by varispeed »

2001zhaozhao wrote: post #43

There's a tradeoff between dense models and MoEs on memory usage vs. compute for the same quality. For example, Qwen3.5 27B and Qwen3.5 122B A10B have similar average performance across benchmarks. The 122B is much faster to run than the 27B (generates more tokens at the same compute). The 27B, on the other hand, uses ~4x less VRAM at low context lengths (less difference at high context lengths). Right now, different…

I still not see the point running these models. I say they produce plausible garbage, nowhere near quality of frontier models (when they work).

Why can't Intel look beyond this nonsense state of affair and build something with 1TB of RAM or more?

What I am trying to say, I am yet to see anything competitive in the market. Cards very much stalled in sub 100GB region and best corporations can do is throw something to run toy models and forget about it after a week.

Re: Intel Arc Pro B70 Review

#60

Post by userbinator »

It should be possible to use the VRAM as extra swap space, when you're not using it for AI or gaming or anything else. 32GB is already more than a lot of computers have as just regular RAM, even sufficient to hold an OS installation:

https://www.tomshardware.com/news/lightweight-windows-11-run...

Post reply on HN