Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Intel announces Arc B-series "Battlemage" discrete graphics with Linux support

phoronix.com

Post reply on HN
Print view View original

571–580 of 722 posts

Re: Intel announces Arc B-series "Battlemage" discrete graphics with Linux support

#571

Post by mahkeiro »

bayindirh wrote: post #358
landryraccoon wrote: post #342

Earlier quoted context omitted.

It's possible you're underestimating the open source community. If there's a competing platform that hobbyists can tinker with, the ecosystem can improve quite rapidly, especially when the competing platform is completely closed and hobbyists basically are locked out and have no alternative.

> It's possible you're underestimating the open source community. On the contrary. You really don't know how I love and prefer open source and love a more leveling playing field. > If there's a competing platform that hobbyists can tinker with... AMD's cards are better from hardware and software architecture standpoint, but the performance is not there yet. Plus, ROCm libraries are not that mature, but they're gettin…

ROCm doesn't really matter when the hardware is almost the same as Nvidia cards. AMD is not selling "cheaper" card with a lot of RAM, what the original poster was asking. (and a reason why people who like to tinker with large model are using Macs).

Re: Intel announces Arc B-series "Battlemage" discrete graphics with Linux support

#572

Post by nine_k »

bayindirh wrote: post #264
treprinum wrote: post #26

Why don't they just release a basic GPU with 128GB RAM and eat NVidia's local generative AI lunch? The networking effect of all devs porting their LLMs etc. to that card would instantly put them as a major CUDA threat. But beancounters running the company would never get such an idea...

Disclosure: HPC admin who works with NIVIDA cards here. Because, no. It's not as simple as that. NVIDIA has a complete ecosystem now. They have cards. They have cards of cards (platforms), which they produce, validate and sell. They have NVLink crossbars and switches which connects these cards on their card of cards with very high speeds and low latency. For inter-server communication they have libraries which coordi…

You are of course right WRT datacenter use of GPUs. The OP spoke about local generation though. It is of course a smaller market, but a market nevertheless, and the more amateurs and students are using your product, the more of them would consider applying it in more professional settings.

Sun (Sparc) and HP (PA-RISC) used to own most of the server market in 1990, but lost most of it to x86 by 2000. Few people had a Sun box with Solaris, but tons of people had access to a PC with Linux, which was inferior in many ways, but well-known and much less locked-up.

Re: Intel announces Arc B-series "Battlemage" discrete graphics with Linux support

#573

Post by coolspot »

jmward01 wrote: post #179
hajile wrote: post #157

Earlier quoted context omitted.

Launching a new SKU for $500-1000 with 48gb of RAM seems like a profitable idea. The GPU isn't top-of-the-line, but the RAM would be unmatched for running a lot of models locally.

give me 48gb with reasonable power consumption so I can dev locally and I will buy it in a heartbeat. Anyone that is fine-tuning would want a setup like that to test things before pushing to real GPUs. And in reality if you can fine-tune on a card like that in two days instead of a few hours it would totally be worth it.

> give me 48gb with reasonable power consumption so I can dev locally and I will buy it in a heartbeat

https://a.co/d/1LMNatf

Re: Intel announces Arc B-series "Battlemage" discrete graphics with Linux support

#574

Post by skirmish »

treprinum wrote: post #137
Muskyinhere wrote: post #114

Earlier quoted context omitted.

No one is running LLMs on consumer NVidia GPUs or apple MacBooks. A dev, if they want to run local models, probably run something which just fits on a proper GPU. For everything else, everyone uses an API key from whatever because its fundamentaly faster. IF a affordable intel GPU would be relevant faster for inferencing, is not clear at all. A 4090 is at least double the speed of Apples GPU.

4090 is 5x faster than M3 Max 128GB according to my tests but it can't even inference LLaMA-30B. The moment you hit that memory limit the inference is suddenly 30x slower than M3 Max. So a basic GPU with 128GB RAM would trash 4090 on those larger LLMs.

Quantized 30B models should run in 24GB VRAM. A quick search found people doing that with good speed: [1]

    I have a 4090, PCIe 3x16, DDR4 RAM.
    
    oobabooga/text-generation-webui
    using exllama
    I can load 30B 4bit GPTQ models and use full 2048 context
    I get 30-40 tokens/s
[1] https://old.reddit.com/r/LocalLLaMA/comments/14gdsxe/optimal...

Re: Intel announces Arc B-series "Battlemage" discrete graphics with Linux support

#575

Post by Kiro »

philistine wrote: post #158
giobox wrote: post #97

Earlier quoted context omitted.

For sure its been a sweet spot for a very long time for budget conscious gamers looking for best balance of price and frame rates, but 1440p optimized parts are nothing new. Both NVidia and AMD make parts that target 1440p display users too, and have done for years. Even previous Intel parts you can argue were tailored for 1080p/1440p use, given their comparative performance deficit at 4k etc. Assuming they retail at…

I'm baffled that PC gamers have decided that 1440p is the endgame for graphics. When I look at a 27-inch 1440p display, I see pixel edges everywhere. It's right at the edge of losing the visibility of individual pixels, since I can't perceive them at 27-inch 2160p, but not quite there yet for desktop distances. Time marches on, and I become ever more separated from gaming PC enthusiasts.

You incorrectly presume that all gamers care about things such as "pixel edges". I think 1080p is fine. Game mechanics always trump fidelity.

Re: Intel announces Arc B-series "Battlemage" discrete graphics with Linux support

#576

Post by Cumpiler69 »

gs17 wrote: post #34
01HNNWZ0MV43FF wrote: post #32

Earlier quoted context omitted.

Judging by the number of 16 GB laptops I see around, 128 GB of RAM would probably cost a bajillion dollars

One of the great things about having a desktop is being able to get that much for under $300 instead of the price of a second laptop.

Desktop PCs RAM don't have the bandwidth and latency required for running AI.

Re: Intel announces Arc B-series "Battlemage" discrete graphics with Linux support

#577

Post by pjmlp »

talldayo wrote: post #367
pjmlp wrote: post #280

Earlier quoted context omitted.

Nope, because you can't get those at the mall downtown, rather have to custom build, ordering from online shops.

Considering how HN professedly struggles to use non-reversible cables, it wouldn't surprise me if someone started to complain about getting skillchecked by a Molex or SATA. And that CPU, with the little gold tab you gotta line up? That's a whopping three ways you could go wrong! Forget about it, might as well just phone Dell or Apple and pay the idiot tax.

Given my experience with PC parts in the past, most folks would be skillchecked when getting any card for their slots, that don't exactly match the motherboard firmware, bus protocol version or speed, even a x.1 variation makes all the difference.

Also better check if the UEFI firmware does understand the nice NVMe SSD brand, version that was bought.

Be sure to not fill all the slots either, or ensure the electromagnetic radiation from each board doesn't cause issues with their neighbour, they might be a possible reason for random freezes.

Custom PCs aren't IKEA, and my time building them is long gone, I have better things to do.

Re: Intel announces Arc B-series "Battlemage" discrete graphics with Linux support

#578

Post by m00x »

treprinum wrote: post #375
m00x wrote: post #350

Earlier quoted context omitted.

It would be a good idea to start with some basic understanding of GPU, and realizing why this can't easily be done.

That's a question M3 Max with its internal GPU already answered. It's not like I didn't do any HPC or CUDA work in the past to be completely clueless about how GPUs work though I haven't created those libraries myself.

What have you implemented in CUDA?

Re: Intel announces Arc B-series "Battlemage" discrete graphics with Linux support

#579

Post by ryao »

elabajaba wrote: post #554
AnthonyMouse wrote: post #498

Earlier quoted context omitted.

> They can't just slap more memory on the board Why not? It doesn't have to be balanced. RAM is cheap. You would get an affordable card that can hold a large model and still do inference e.g. 4x faster than a CPU. The 128GB card doesn't have to do inference on a 128GB model as fast as a 16GB card does on a 16GB model, it can be slower than that and still faster than any cost-competitive alternative at that size. The…

To get 128GB of RAM on a GPU you'd need at least a 1024 bit bus. GDDR6x is 16Gbit 32 pins, so you'd need 64 GDDR6x chips, which good luck even trying to fit that around the GPU die since traces need to be the same length, and you want to keep them as short as possible. There's also a good chance you can't run a clamshell setup so you'd have to double the bus width to 2048 because 32 GDDR6x chips would kick off way to…

You do not need a 1024-bit bus to put 128GB of some DDR variant on a GPU. You could do a 512-bit bus with dual rank memory. The 3090 had a 384-bit bus with dual rank memory and going to 512-bit from that is not much of a leap.

This assumes you use 32Gbit chips, which will likely be available in the near future. Interestingly, the GDDR7 specification allows for 64Gbit chips:

> the GDDR7 standard officially adds support for 64Gbit DRAM devices, twice the 32Gbit max capacity of GDDR6/GDDR6X

https://www.anandtech.com/show/21287/jedec-publishes-gddr7-s...

Re: Intel announces Arc B-series "Battlemage" discrete graphics with Linux support

#580

Post by imtringued »

AnthonyMouse wrote: post #486
dragontamer wrote: post #285

Earlier quoted context omitted.

Because LPDDR5x is soldered on RAM. Everyone else wants configurable RAM that scales both down (to 16GB) and up (to 2TB), to cover smaller laptops and bigger servers. GPUs with soldered on RAM has 500GB/sec bandwidths, far in excess of Apples chips. So the 8GB or 16GB offered by NVidia or AMD is just far superior at vid o game graphics (where textures are the priority)

The question is why Intel GPUs , which already have soldered memory, aren't sold with more of it. The market here isn't something that can beat enterprise GPUs at training, it's something that can beat desktop CPUs at inference with enough VRAM to fit large models at an affordable price.

Intel also has lunar lake CPUs with on package RAM. They could have added more memory channels like Apple did.
Post reply on HN