Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Intel announces Arc B-series "Battlemage" discrete graphics with Linux support

phoronix.com

Post reply on HN
Print view View original

341–350 of 722 posts

Re: Intel announces Arc B-series "Battlemage" discrete graphics with Linux support

#341

Post by postalrat »

bayindirh wrote: post #264
treprinum wrote: post #26

Why don't they just release a basic GPU with 128GB RAM and eat NVidia's local generative AI lunch? The networking effect of all devs porting their LLMs etc. to that card would instantly put them as a major CUDA threat. But beancounters running the company would never get such an idea...

Disclosure: HPC admin who works with NIVIDA cards here. Because, no. It's not as simple as that. NVIDIA has a complete ecosystem now. They have cards. They have cards of cards (platforms), which they produce, validate and sell. They have NVLink crossbars and switches which connects these cards on their card of cards with very high speeds and low latency. For inter-server communication they have libraries which coordi…

Lets see how quickly that changes if intel releases cards with massive amounts of ram for a fraction of the cost.

Re: Intel announces Arc B-series "Battlemage" discrete graphics with Linux support

#342

Post by landryraccoon »

bayindirh wrote: post #307
throwaway48476 wrote: post #288

Earlier quoted context omitted.

All of that is highly relevant for training but what the poster was asking for is a desktop inference card.

You use at least half of this stack for desktop setups. You need copying daemons, the ecosystem support (docker-nvidia, etc.), some of the libraries, etc. even when you're on a single system. If you're doing inference on a server; MIG comes into play. If you're doing inference on a larger cloud, GPU-direct storage comes into play. It's all modular.

It's possible you're underestimating the open source community.

If there's a competing platform that hobbyists can tinker with, the ecosystem can improve quite rapidly, especially when the competing platform is completely closed and hobbyists basically are locked out and have no alternative.

Re: Intel announces Arc B-series "Battlemage" discrete graphics with Linux support

#343

Post by ethbr1 »

ksec wrote: post #306
wtallis wrote: post #53

Earlier quoted context omitted.

Just how "basic" do you think a GPU can be while having the capability to interface with that much DRAM? Getting there with GDDR6 would require a really wide memory bus even if you could get it to operate with multiple ranks. Getting to 128GB with LPDDR5x would be possible with the 256-bit bus width they used on the top parts of the last generation, but would result in having half the bandwidth of an already mediocre…

Thank You Wtallis. Somewhere along the line, this basic "knowledge" of hardware is completely lost. I dont expect this to be explained in any comment section on old Anandtech. It seems hardware enthusiast has mostly disappeared, I guess that is also why Anandtech closed. We now live in a world where most site are just BS rumours.

That's because Anand Lal Shimpi is a CompE by training.

Not too many hardware enthusiast site editors have that academic background.

And while fervor can sometimes substitute for education... probably not in microprocessor / system design.

Re: Intel announces Arc B-series "Battlemage" discrete graphics with Linux support

#344

Post by throwaway48476 »

landryraccoon wrote: post #342
bayindirh wrote: post #307

Earlier quoted context omitted.

You use at least half of this stack for desktop setups. You need copying daemons, the ecosystem support (docker-nvidia, etc.), some of the libraries, etc. even when you're on a single system. If you're doing inference on a server; MIG comes into play. If you're doing inference on a larger cloud, GPU-direct storage comes into play. It's all modular.

It's possible you're underestimating the open source community. If there's a competing platform that hobbyists can tinker with, the ecosystem can improve quite rapidly, especially when the competing platform is completely closed and hobbyists basically are locked out and have no alternative.

Innovation is a bottom up process. If they sell the hardware the community will spring up to take advantage.

Re: Intel announces Arc B-series "Battlemage" discrete graphics with Linux support

#345

Post by jjmarr »

treprinum wrote: post #309
p1esk wrote: post #257

Earlier quoted context omitted.

We would also need to imagine AMD fixing their software.

I think plenty of enthusiastic open source devs would jump at it and fix their software if the software was reasonably open. The same effect as what happened when Meta released LLaMA.

It is open and they regularly merge PRs.

https://github.com/ROCm/ROCm/pulls?q=is%3Apr+is%3Aclosed

Re: Intel announces Arc B-series "Battlemage" discrete graphics with Linux support

#346

Post by jmward01 »

bayindirh wrote: post #264
treprinum wrote: post #26

Why don't they just release a basic GPU with 128GB RAM and eat NVidia's local generative AI lunch? The networking effect of all devs porting their LLMs etc. to that card would instantly put them as a major CUDA threat. But beancounters running the company would never get such an idea...

Disclosure: HPC admin who works with NIVIDA cards here. Because, no. It's not as simple as that. NVIDIA has a complete ecosystem now. They have cards. They have cards of cards (platforms), which they produce, validate and sell. They have NVLink crossbars and switches which connects these cards on their card of cards with very high speeds and low latency. For inter-server communication they have libraries which coordi…

Most of the above infra is predicated on limiting RAM so that you need so much communication between cards. Bump the RAM up and you could do single card inference and all those connections become overhead that could have gone to more ram. For training there is an argument still, but even there the more RAM you have the less all that connectivity gains you. RAM has been used to sell cards and servers for a long time now, it is time to open the floodgates.

Re: Intel announces Arc B-series "Battlemage" discrete graphics with Linux support

#347

Post by serf »

dimensi0nal wrote: post #172
bongodongobob wrote: post #152

Earlier quoted context omitted.

I don't know a single person in real life that has any desire to run local LLMs. Even amongst my colleagues and tech friends, not very many use LLMs period. It's still very niche outside AI enthusiasts. GPT is better than anything I can run locally anyway. It's not as popular as you think it is.

The only consumer demand for local AI models is for generating pornography

absolutely wrong -- if you're not clever enough to think of any other reason to run an LLM locally then don't condemn the rest of the world to "well they're just using it for porno!"

Re: Intel announces Arc B-series "Battlemage" discrete graphics with Linux support

#348

Post by m00x »

treprinum wrote: post #159
m00x wrote: post #79

Earlier quoted context omitted.

FLUX.1 D generation is about a minute at 20 steps on a 4080, but takes 35 minutes on the CPU.

4080 won't do video due to low RAM. The GPU doesn't have to be as fast there, it can be 5x slower which is still way faster than a CPU. And Intel can iterate from there.

It won't be 5x slower, it would be 20-50x slower if you would implement it as you said.

You can't just "add more ram" to GPUs and have them work the same way. Memory access is completely different than on CPUs.

Re: Intel announces Arc B-series "Battlemage" discrete graphics with Linux support

#349

Post by refulgentis »

throwaway48476 wrote: post #329
tokioyoyo wrote: post #144

Earlier quoted context omitted.

This is the weird part, I saw the same comments in other threads. People keep saying how everyone yearns for local LLMs… but other than hardcore enthusiasts it just sounds like a bad investment? Like it’s a smaller market than gaming GPUs. And by the time anyone runs them locally, you’ll have bigger/better models and GPUs coming out, so you won’t even be able to make use of them. Maybe the whole “indoctrinate users t…

Intel sold their GPUs at negative margin which is part of why the stock fell off a cliff. If they could double the vram they could raise the price into the green even selling thousands, likely closer to 100k, would be far better than what they're doing now. The problem is Intel is run by incompetent people who guard their market segments as tribal fiefs instead of solving for the customer.

By subsidizing it more they'll lose less money?

Re: Intel announces Arc B-series "Battlemage" discrete graphics with Linux support

#350

Post by m00x »

treprinum wrote: post #170
m00x wrote: post #87

Earlier quoted context omitted.

Because the CPU has to load the model in parts for every cycle so you're spending a lot of time on IO and it offsets processing. You're talking about completely different things here. It's fine if you're doing a few requests at home, but if you're actually serving AI models, CUDA is the only reasonable choice other than ASICs.

My comment was about Intel having a starter project, getting enthusiastic response from devs, network effects and iterate from there. They need a way to threaten Nvidia and just focusing on what they can't do won't bring them there. There is one route where they can disturb Nvidia's high end over time and that's a cheap basic GPU with lots of RAM. Like Ryzen 1st gen whose single core performance was two generations b…

It would be a good idea to start with some basic understanding of GPU, and realizing why this can't easily be done.
Post reply on HN