Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Intel announces Arc B-series "Battlemage" discrete graphics with Linux support

phoronix.com

Post reply on HN
Print view View original

621–630 of 722 posts

Re: Intel announces Arc B-series "Battlemage" discrete graphics with Linux support

#621

Post by ben_w »

hulitu wrote: post #278
choilive wrote: post #198

Earlier quoted context omitted.

I think the more accurate denominator would be the world population. People are seeing benefits to LLMs even outside of the office.

> I think the more accurate denominator would be the world population. People are seeing benefits to LLMs even outside of the office. For example ? (besides deep fakes)

I use one in the kitchen, because it's easier than the ads and prose on most recipe websites, and it can adapt to whatever ingredients I actually have rather than being a fixed list.

Used them in the garden while weeding and in the garden store while planning what to plant, in both cases to identify plants by image and tell me about them — though I'd say image capable AI are no longer mere "large language models".

Used ChatGPT while shopping to help me locate products in the store I was in, when I couldn't find just by wandering the isles, by uploading a photo of the aisle I happened to be in at the point I gave up.

Re: Intel announces Arc B-series "Battlemage" discrete graphics with Linux support

#622

Post by ryao »

ac29 wrote: post #448
treprinum wrote: post #195

Earlier quoted context omitted.

That's why I said "basic GPU". It doesn't have to be too fast but it should still be way faster than a regular CPU. Intel already has Xeon Phi so a lot of things were developed already (like memory controller, heavy parallel dies etc.)

I wonder at that point you'd just be better served by CPU with 4 channels of RAM. If my math is right 4 channels of DDR5-8000 would get you 256GB/s. Not as much bandwidth as a typical discrete GPU, but it would be trivial to get many hundreds of GB of RAM and would be expandable. Unfortunately I don't think either Intel or AMD makes a CPU that supports quad channel RAM at a decent price.

4 channels of DDR5-8000 would give you 128GB/sec. DDR5 has 2x32-bit channels per DIMM rather than 1x64-bit channel like DDR4 did. You would need 8 channels of DDR5-8000 to reach 256GB/sec.

Re: Intel announces Arc B-series "Battlemage" discrete graphics with Linux support

#623

Post by lhl »

Recently I did some testing of the IPEX-LLM llama.cpp backend on LNL's Xe2: https://www.reddit.com/r/LocalLLaMA/comments/1gheslj/testing...

Based on scaling by XMX/engine clock napkin math, the B580 should have 230 FP16 TFLOPS and 456 GB/s MBW theoretical. At similar efficiency to LNL Xe2, that should be about pp512 ~4700 t/s and tg128 ~77 t/s for a 7B class model. This would be about 75% of a 3090 for pp and 50% for tg (and of course, 50% of memory). For $250, that's not too bad.

I do want to note a couple things from my poking around. The IPEX-LLM [1] was very responsive, and was able to address an issue I had w/ llama.cpp within days. They are doing weekly update releases, so that's great. The IPEX stands for Intel Extension for PyTorch [2] and it is a mostly drop-in for PyTorch: "Intel® Extension for PyTorch* extends PyTorch* with up-to-date features optimizations for an extra performance boost on Intel hardware. Optimizations take advantage of Intel® Advanced Vector Extensions 512 (Intel® AVX-512) Vector Neural Network Instructions (VNNI) and Intel® Advanced Matrix Extensions (Intel® AMX) on Intel CPUs as well as Intel Xe Matrix Extensions (XMX) AI engines on Intel discrete GPUs. Moreover, Intel® Extension for PyTorch* provides easy GPU acceleration for Intel discrete GPUs through the PyTorch* xpu device."

All of this depends on Intel oneAPI Base Kit [3] which has easy Linux (and presumably Windows) support. I am normally an AUR guy on my Arch Linux workstation, but those are basically broken and I had much more success installing oneAPI Base Kit (w/o issues) directly in Arch Linux. Sadly, this is also where there are issues some of the code is either dependent on older versions of oneAPI Base Kit that are no longer available (vLLM requires oneAPI Base Toolkit 2024.1 - this is not available for download from the Intel site anymore) or in dependency hell (GPU whisper simply will not work, ipex-llm[xpu] has internal conflicts from the get go), so it's not all sunshine. On average, ROCm w/ RNDA3 is much more mature (while not always the fastest, most basic things do just work now).

[1] https://github.com/intel-analytics/ipex-llm

[2] https://github.com/intel/intel-extension-for-pytorch

[3] https://www.intel.com/content/www/us/en/developer/tools/onea...

Re: Intel announces Arc B-series "Battlemage" discrete graphics with Linux support

#624

Post by ryao »

FeepingCreature wrote: post #606
ac29 wrote: post #448

Earlier quoted context omitted.

I wonder at that point you'd just be better served by CPU with 4 channels of RAM. If my math is right 4 channels of DDR5-8000 would get you 256GB/s. Not as much bandwidth as a typical discrete GPU, but it would be trivial to get many hundreds of GB of RAM and would be expandable. Unfortunately I don't think either Intel or AMD makes a CPU that supports quad channel RAM at a decent price.

I think all the people saying "just use a CPU" massively underestimate the speed difference between current CPUs and current GPUs. There's like four orders of magnitude. It's not even in the same zip code. Say you have a 64-core CPU at 2Ghz with 512-bit 1-cycle FP16 instructions. That gives you 32 ops per cycle, 2048 across the entire package, so 4TFlops. My 7900 XTX does 120TFlops. To match that, you would need to s…

Usually, you can do 2 AVX-512 operations per cycle and using FMADD (fused multiply-add) instructions, you can do two floating point operations for the price of one. That would be 128 operations per cycle per core. The result would be 16TFlops on a 2GHz 64 core CPU, not 4 TFlops. This would give a 1 order of magnitude difference, rather than 4 orders of magnitude.

For inference, prompt processing is compute intensive, while token generation is memory bandwidth bound. The differences in memory bandwidth between CPUs and GPUs tend to be more profound than the difference in compute.

Re: Intel announces Arc B-series "Battlemage" discrete graphics with Linux support

#625

Post by ryao »

FeepingCreature wrote: post #608
talldayo wrote: post #166

Earlier quoted context omitted.

> And indeed that's exactly what will be required for any serious attempt to cut into their monopoly position. You misunderstand why and how Nvidia is a monopoly. Many companies make GPUs, and all those GPUs can be used for computation if you develop compute shaders for them. This part is not the problem, you can already go buy cheaper hardware that outperforms Nvidia if price is your only concern. Software is the is…

CUDA is not the issue. AMD have already reimplemented like 80% of it, and honestly that part of it mostly works fine. Pytorch supports it, (almost) all the big frameworks support it, if you're not doing really arcane things it just works. It's the drivers! They took like two years after the release of their flagship card to stop randomly crashing. Everything geohot has ever said about AMD drivers is 100% true. They j…

What did he say?

Re: Intel announces Arc B-series "Battlemage" discrete graphics with Linux support

#626

Post by ryao »

Numerlor wrote: post #65
jsheard wrote: post #51

Earlier quoted context omitted.

> The IO requires a lot of die space. And even if you spend a lot of die space on memory controllers, you can only fit so many GDDR chips around the GPU core while maintaining signal integrity. HBM sidesteps that issue but it's still too expensive for anything but the highest end accelerators, and the ordinary LPDDR that Apple uses is lacking in bandwidth compared to GDDR, so they have to compensate with ginormous am…

Going off of how the 4090 and 7900 xtx is arranged I think you could maybe fit on or two chips more around the die over their 12, but that's still a far cry from 128. That would probably just need a shared bus like normal DDR as you're not fitting that much with 16 gbit density

Look at the 3090, which uses 24 chips (12 on one side and 12 on another). Pushing it to 32 is doable. 32 is all you need to reach 128GB VRAM with the 32Gbit GDDR7 chips that should be on the market in the near future.

Re: Intel announces Arc B-series "Battlemage" discrete graphics with Linux support

#627

Post by rocqua »

philistine wrote: post #158
giobox wrote: post #97

Earlier quoted context omitted.

For sure its been a sweet spot for a very long time for budget conscious gamers looking for best balance of price and frame rates, but 1440p optimized parts are nothing new. Both NVidia and AMD make parts that target 1440p display users too, and have done for years. Even previous Intel parts you can argue were tailored for 1080p/1440p use, given their comparative performance deficit at 4k etc. Assuming they retail at…

I'm baffled that PC gamers have decided that 1440p is the endgame for graphics. When I look at a 27-inch 1440p display, I see pixel edges everywhere. It's right at the edge of losing the visibility of individual pixels, since I can't perceive them at 27-inch 2160p, but not quite there yet for desktop distances. Time marches on, and I become ever more separated from gaming PC enthusiasts.

I find dual 24 inch 1440p q great compromise. Higher pixel density, decent amount of screen real estate, and nice to have an auxiliary monitor when gaming.

I run the second monitor off the IGPU so it doesn't even tax the main GPU.

Re: Intel announces Arc B-series "Battlemage" discrete graphics with Linux support

#628

Post by Alifatisk »

tommica wrote: post #276

Probably would jump to Intel once my 3060 gets too old

You have a long time to go, unless you want to play the latest and greatest AAA games, but that will not be your cards fault, it's the game studios not optimizing their game.

Re: Intel announces Arc B-series "Battlemage" discrete graphics with Linux support

#629

Post by ryao »

treprinum wrote: post #381
phkahler wrote: post #365

Earlier quoted context omitted.

You can run an AMD APU with 128GB of shared RAM.

It's too slow and not very compatible. Most BIOSes also don't allow sharing that much memory with GPU (max like 16GB).

Isn’t that setting just a historical thing annd ann integrated GPU is able to access any system memory that is mapped by the IOMMU? I assume this is how it works for people using the NVIDIA Jetson AGX Orin 64GB Developer Kit to do inference. I do not know why it would be different for AMD APUs.

Re: Intel announces Arc B-series "Battlemage" discrete graphics with Linux support

#630

Post by FeepingCreature »

ryao wrote: post #625
FeepingCreature wrote: post #608

Earlier quoted context omitted.

CUDA is not the issue. AMD have already reimplemented like 80% of it, and honestly that part of it mostly works fine. Pytorch supports it, (almost) all the big frameworks support it, if you're not doing really arcane things it just works. It's the drivers! They took like two years after the release of their flagship card to stop randomly crashing. Everything geohot has ever said about AMD drivers is 100% true. They j…

What did he say?

Geohot (temporarily) giving up. https://github.com/ROCm/ROCm/issues/2198#issuecomment-157438... Sadly most of the real spicy Twitter messages are gone since he deleted all his content, but there was a really fun one where he went off on a beautifully cryptic commit message in the driver. He also begged AMD to opensource the firmware so he could debug it. Sadly, AMD promised to do it and then nothing happened, as is typical for AMD promises. That's why tinygrad nowadays is aiming to just bypass the driver and firmware entirely.
Post reply on HN