Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Intel announces Arc B-series "Battlemage" discrete graphics with Linux support

phoronix.com

Post reply on HN
Print view View original

611–620 of 722 posts

Re: Intel announces Arc B-series "Battlemage" discrete graphics with Linux support

#611

Post by ryao »

dgfitz wrote: post #286
bayindirh wrote: post #264

Earlier quoted context omitted.

Disclosure: HPC admin who works with NIVIDA cards here. Because, no. It's not as simple as that. NVIDIA has a complete ecosystem now. They have cards. They have cards of cards (platforms), which they produce, validate and sell. They have NVLink crossbars and switches which connects these cards on their card of cards with very high speeds and low latency. For inter-server communication they have libraries which coordi…

How does any of this make money?

HPC is used in research, which is often not expected to make money. The hope is that the research will result in something that makes money. One example would be drug discovery. Another would be weather prediction, which is not so much a way to make money, but to minimize losses.

Re: Intel announces Arc B-series "Battlemage" discrete graphics with Linux support

#612

Post by treprinum »

jjmarr wrote: post #485
treprinum wrote: post #385

Earlier quoted context omitted.

AMD GPUs aren't very attractive to ML folks because they don't outshine Nvidia in any single aspect. Blasting lots of RAM onto a GPU would make it attractive immediately with lots of attention from devs occupied with more interesting things.

MI300X already leads in VRAM as it has 192 GB. For local inference, 7900 XTX has 24 GB of VRAM for less than $1000. At what threshold of VRAM would you start being interested in MI?

Problem with MI300x is the price. Problem with 7900XTX is that it's at best as good as Nvidia with the same RAM for a similar price. If 7900XTX had e.g. 64GB of RAM, was 2x slower than 4080, and kept its price, it would sell like crazy.

Re: Intel announces Arc B-series "Battlemage" discrete graphics with Linux support

#613

Post by aryonoco »

kridsdale1 wrote: post #35
rmm wrote: post #9

I put an a360 Card into an old machine I turned into a plex server. It turned it into a transcoding powerhouse. I can do multiple indepdent streams now without it skipping a beat. Price-performance ratio was off the chart

Any idea how that compares to Apple Silicon for that job? I bought the $599 MacBook Air with M1 as my plex server for this reason. Transcodes 4k HEVC and doesn’t even need a fan. Sips watts.

Apple's hardware encode/decode for AV1 is quite literally, shit.

Re: Intel announces Arc B-series "Battlemage" discrete graphics with Linux support

#614

Post by ryao »

bayindirh wrote: post #295
dgfitz wrote: post #286

Earlier quoted context omitted.

How does any of this make money?

When this walled garden is the only way to use GPUs with high efficiency and everybody is using this stack, and NVIDIA controlling the supply of these "platform boards" to OEMs, they don't make money, but they literally print it. However, AMD is coming for them because a couple of high profile supercomputer centers (LUMI, Livermore, etc.) are using Instinct cards and pouring money to AMD to improve their cards and st…

I think he was asking how HPC makes money. The answer as far as I know is that it often does not for anyone other than the vendors.

I wonder where you got your information on AMD’s “Linux driver architecture”. It is reportedly a mess:

https://news.ycombinator.com/item?id=34832660

So far, I have been very happy with Nvidia’s Linux drivers.

Re: Intel announces Arc B-series "Battlemage" discrete graphics with Linux support

#615

Post by treprinum »

skirmish wrote: post #574
treprinum wrote: post #137

Earlier quoted context omitted.

4090 is 5x faster than M3 Max 128GB according to my tests but it can't even inference LLaMA-30B. The moment you hit that memory limit the inference is suddenly 30x slower than M3 Max. So a basic GPU with 128GB RAM would trash 4090 on those larger LLMs.

Quantized 30B models should run in 24GB VRAM. A quick search found people doing that with good speed: [1] I have a 4090, PCIe 3x16, DDR4 RAM. oobabooga/text-generation-webui using exllama I can load 30B 4bit GPTQ models and use full 2048 context I get 30-40 tokens/s [1] https://old.reddit.com/r/LocalLLaMA/comments/14gdsxe/optimal...

Quantized sure but there is some loss of variability of the output one can notice quickly with 30B models. If you want to use the fp16 version you are out of luck.

Re: Intel announces Arc B-series "Battlemage" discrete graphics with Linux support

#616

Post by [deleted] »

ryao wrote: post #544
derefr wrote: post #535

Earlier quoted context omitted.

I presume the counterargument is that inference hosting is commoditized (sort of like how stateless CPU-based containerized workload hosts are commoditized); there’s no margin in that business, because it is parallelizable, and arbitrarily schedulable, and able to be spread across heterogenous hardware pretty easily (just route individual requests to sub-cluster A or B), preventing any kind of lock-in and thus any ki…

Intel does make cards aimed at this space too: https://www.intel.com/content/www/us/en/products/details/pro... Coincidentally, it has 128GB of RAM. However, it is not a GPU, is designed to do training too and uses expensive HBM. Modern GPUs can do more than inference/training and the original poster asked about a GPU with 128GB of RAM, not a card that can only do inferencing as you described. Interestingly, Qualcomm…

[deleted]

Re: Intel announces Arc B-series "Battlemage" discrete graphics with Linux support

#617

Post by ryao »

treprinum wrote: post #357
amelius wrote: post #209

Earlier quoted context omitted.

What if they put 8 identical GPUs in the package, each with 1/8 the memory? Would that be a useful configuration for a modern LLM?

K80 used to be two glued K40 but their interconnect was barely faster than PCIe so it didn't have much benefit as one had to move stuff between two internal GPUs anyway.

Inference workloads likely won’t care very much. For llama 3.1 405B with bf16 when you split the workload across GPUs by layer, you need to do a 32KB memory copy before the next GPU can begin processing. That can be done incredibly quickly over PCI-E.

Re: Intel announces Arc B-series "Battlemage" discrete graphics with Linux support

#618

Post by ryao »

chessgecko wrote: post #54
treprinum wrote: post #26

Why don't they just release a basic GPU with 128GB RAM and eat NVidia's local generative AI lunch? The networking effect of all devs porting their LLMs etc. to that card would instantly put them as a major CUDA threat. But beancounters running the company would never get such an idea...

GDDR isnt like the ram that connects to cpu, it's much more difficult and expensive to add more. You can get up to 48GB with some expensive stacked gddr, but if you wanted to add more stacks you'd need to solve some serious signal timing related headaches that most users wouldn't benefit from. I think the high memory local inference stuff is going to come from "AI enabled" cpus that share the memory in your computer.…

It is not stacked. It is multirank. Stacking means putting multiple layers on the same chip. They are already doing it for HBM. They will likely do it for other forms of DRAM in the future. Samsung reportedly will begin doing it in the 2030s:

https://www.tomshardware.com/pc-components/dram/samsung-outl...

I am not sure why they can already do stacking for HBM, but not GDDR and DDR. My guess is that it is cost related. I have heard that HBM reportedly costs 3 times more than DDR. Whatever they are doing to stack it now that is likely much more expensive than their planned 3D fabrication node.

Re: Intel announces Arc B-series "Battlemage" discrete graphics with Linux support

#619

Post by mrweasel »

hulitu wrote: post #207
bryanlarsen wrote: post #138

Earlier quoted context omitted.

These are $200 low end cards, the B5X0 cards. Presumably they have B7X0 and perhaps even B9X0 cards in the pipeline as well.

> These are $200 low end cards Hm, i wouldn't consider 200$ low end.

I was watching a few of the preliminary commentaries on the Battlemage cards, e.g. Linus Tech Tips and so on, and they said the same thing "focus on the low/mid-end". My conclusion is that what they are talking about is "low end, brand new, current gen. graphics card". It's a very special type of low end.

The Intel cards are getting more interesting for me as I'm questioning my continued use of macOS. Intels focus on Linux support makes their options really interesting, though I don't see a need for something as powerful as these new cards.

Re: Intel announces Arc B-series "Battlemage" discrete graphics with Linux support

#620

Post by schmorptron »

confident_inept wrote: post #13

I'm really curious to see if these still rely heavily on resizable BAR. Putting these in old computers in linux without reBAR support makes the driver crash with literally any load rendering the cards completely unusable. It's a real shame, the single slot a380 is a great performance for price light gaming and general use card for small machines.

Yes, it even says so as a requirement on the box.

What I'm also interested in is whether they'll require a PCIE 4.0 motherboard or if a 3.0 x16 slot is fine.

Also whether their software properly supports VR now...

Post reply on HN