Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Intel announces Arc B-series "Battlemage" discrete graphics with Linux support

phoronix.com

Post reply on HN
Print view View original

511–520 of 722 posts

Re: Intel announces Arc B-series "Battlemage" discrete graphics with Linux support

#511

Post by rubatuga »

bongodongobob wrote: post #152
Implicated wrote: post #101

12GB memory -.- I feel like _anyone_ who can pump out GPU's with 24GB+ of memory that are capable to use for py-stuff would benefit greatly. Even if it's not as performant as the NVIDIA options - just to be able to get the models to run, at whatever speed. They would fly off the shelves.

I don't know a single person in real life that has any desire to run local LLMs. Even amongst my colleagues and tech friends, not very many use LLMs period. It's still very niche outside AI enthusiasts. GPT is better than anything I can run locally anyway. It's not as popular as you think it is.

I run a 12GB model on my 3060 and use it to help answer healthcare questions. I'm currently doing a medical residency. (No I don't use it to diagnose). It helps comply with any HIPAA style regulations. I sometimes use it to fix up my emails. Not sure why people are longing for a 128GB card, just download a quantized model and run with LM Studio (https://lmstudio.ai/). At least two of my colleagues are using ChatGPT on a regular basis. LLMs are being used in the ER department. LLMs and speech models are being used in psychiatry visits.

Re: Intel announces Arc B-series "Battlemage" discrete graphics with Linux support

#512

Post by to11mtm »

bayindirh wrote: post #358
landryraccoon wrote: post #342

Earlier quoted context omitted.

It's possible you're underestimating the open source community. If there's a competing platform that hobbyists can tinker with, the ecosystem can improve quite rapidly, especially when the competing platform is completely closed and hobbyists basically are locked out and have no alternative.

> It's possible you're underestimating the open source community. On the contrary. You really don't know how I love and prefer open source and love a more leveling playing field. > If there's a competing platform that hobbyists can tinker with... AMD's cards are better from hardware and software architecture standpoint, but the performance is not there yet. Plus, ROCm libraries are not that mature, but they're gettin…

IDK, I found a post that's 2 years old that has links to doing llama and SD on an Arc [0] (although might be linux only), I feel like a cheap huge ram card would create a 'critical mass' as far as being able to start optimizing, and then from a longer term Intel could promise and deliver on 'scale up' improvements.

It would be a huge shift for them. To go from preferring some (sometimes not quite reached) metric, to, perhaps rightly play the 'reformed underdog'. Commoditize Big-Memory ML Capable GPUs, even if they aren't quite as competitive as the top players at first.

Will the other players respond? Yes. But ruin their margin. I know that sounds cutthroat[1] but hey I'm trying to hypothetically sell this to whomever is taking the reigns after Pat G.

> NVIDIA gives a lot of support to universities, researchers and institutions who play with their cards. Big cards may not be free, but know-how, support and first steps are always within reach. Plus, their researchers dogfood their own cards, and write papers with them.

Ideally they need to do that too. Ideally they have some 'high powered' prototypes (e.x. lets say they decide a 2-gpu per card design with an interlink is feasible for some reason) to share as well. This may not be be entirely ethical[1] in this example of how a corp could play it out, again it's a thought experiment since intel has NOT announced or hinted at a larger memory card anyway.

> AMD also needs to be able to enable ROCm on desktop properly, so people can start hacking it at home

AMD's driver story has always been a hot mess, My desktop won't behave with both my onboard video and 4060 enabled, every AMD card I've had winds up with some weird firmware quirk one way or another... I guess I'm saying their general level of driver quality doesn't lend to hope they'll fix dev tools that soon...

[0] - https://old.reddit.com/r/LocalLLaMA/comments/12khkka/running...

[1] - As you said, it's about winning and it can get ugly.

Re: Intel announces Arc B-series "Battlemage" discrete graphics with Linux support

#513

Post by barkingcat »

p1esk wrote: post #262
jsheard wrote: post #248

Earlier quoted context omitted.

Because Apple isn't playing the same game as everyone else. They have the money and clout to buy out TSMCs bleeding-edge processes and leave everyone else with the scraps, and their silicon is only sold in machines with extremely fat margins that can easily absorb the BOM cost of making huge chips on the most expensive processes money can buy.

Bleeding edge processes is what Intel specializes in. Unlike Apple, they don’t need TSMC. This should have been a huge advantage for Intel. Maybe that’s why Gelsinger got the boot.

if you think intel has bleeding edge processes, that hasn't been the case for over 6 years ...

Re: Intel announces Arc B-series "Battlemage" discrete graphics with Linux support

#514

Post by barkingcat »

anon373839 wrote: post #495
jsheard wrote: post #270

Earlier quoted context omitted.

Intel's foundry side has been floundering so hard that they've resorted to using TSMC themselves in an attempt to keep up with AMD. Their recently launched CPUs are a mix of Intel-made and TSMC-made chiplets, but the latter accounts for most of the die area.

I'm not certain this is quite as damning as it sounds. My understanding is that the foundry business was intentionally walled off from the product business, and that the latter wasn't going to be treated as a privileged customer.

no, in fact, it sounds even more damning because client side was able to pick whatever was best on the market, and it wasn't intel. Client side could to learn and customize their designs to use another company's processes (this is an extremely hard thing to do by the way) faster than intel foundry could even get their pants up in the morning.

Intel foundry screwed up so badly that Nokia's server division was almost shut down because of Intel Foundry's failure. (imagine being so bad at your job, that your clients go out of business) If Intel client side chose to use Foundry, there just wouldn't be any chips to sell.

Re: Intel announces Arc B-series "Battlemage" discrete graphics with Linux support

#515

Post by ryao »

wtallis wrote: post #146
treprinum wrote: post #115

Earlier quoted context omitted.

Nothing in my comment says about pricing it at the M4 Max level. Apple charges as much because they can (typing this on an $8000 M3 Max). 128GB LPDDR5 is dirt cheap these days just Apple adds its premium because they like to. Nothing prevents Intel from releasing a basic GPU with that much RAM for under $1k.

You're asking for a GPU die at least as large as NVIDIA's TU102 that was $1k in 2018 when paired with only 11GB of RAM (because $1k couldn't get you a fully-enabled die to use 12GB of RAM). I think you're off by at least a factor of two in your cost estimates.

If Intel came out with an ARC GPU with 128GB VRAM at a $2000 price point, I and many others would likely buy it immediately.

Re: Intel announces Arc B-series "Battlemage" discrete graphics with Linux support

#516

Post by rubatuga »

Tuna-Fish wrote: post #212
hajile wrote: post #157

Earlier quoted context omitted.

Launching a new SKU for $500-1000 with 48gb of RAM seems like a profitable idea. The GPU isn't top-of-the-line, but the RAM would be unmatched for running a lot of models locally.

It's not technically possible to just slap on more RAM. GDDR6 is point-to-point with option for clamshell, and the largest chips in mass production are 16Gbit/32 bit. So, for a 192bit card, the best you can get is 192/32×16Gbit×2 = 24GB. To have more memory, you have to design a new die with a wider interface. The design+test+masks on leading edge silicon is tens of millions of NRE, and has to be paid well over a yea…

Interesting, will have to try it out when it is released.

Re: Intel announces Arc B-series "Battlemage" discrete graphics with Linux support

#517

Post by barkingcat »

michaelmrose wrote: post #438
JBiserkov wrote: post #369

Earlier quoted context omitted.

> and their silicon is only sold in machines with extremely fat margins Like the brand new Mini that cost 600 USD and went to 500 during Black week.

The 600 mini is on the m4 with the anemic 10 core GPU. This is for grandama or your 10 year old. The good one which is still slower than m4 max is 2200. If you want the max you need at least a macbook pro starting at 3200 and if you want the better one with 128G RAM it starts at about 5k

No matter how few GPU cores the M4 has, it is still an extremely potent product on the whole.

Better than most of the pc's out there.

Re: Intel announces Arc B-series "Battlemage" discrete graphics with Linux support

#518

Post by wtallis »

ryao wrote: post #506
wtallis wrote: post #53

Earlier quoted context omitted.

Just how "basic" do you think a GPU can be while having the capability to interface with that much DRAM? Getting there with GDDR6 would require a really wide memory bus even if you could get it to operate with multiple ranks. Getting to 128GB with LPDDR5x would be possible with the 256-bit bus width they used on the top parts of the last generation, but would result in having half the bandwidth of an already mediocre…

It is possible to have multiple memory ranks to reduce the bus width requirements for a given amount of memory. Nvidia has demonstrated that this is doable with GDDR6X on the RTX 3090. The RTX 3090 has a 384-bit bus with 24 memory ICs, despite only needing 12 to reach 384-bit. That means it has every two chips sharing one 32-bit interface, which is a dual rank configuration. If you look at the history of computer mem…

Having four chips per channel is exactly why this is implausible. DDR5 can barely operate with four ranks per channel, at severely reduced speeds. Pulling that off with GDDR6 or GDDR7 is not something we can presume to be possible without specific evidence. The highest-density configurations possible for LPDDR5x are dual-rank and byte mode (one chip per 8 bits of the memory bus, so two chips ganged together to populate a 16-bit channel) — and that still operates at less than half the speed of GDDR6.

I've not seen any proposals for buffering LPDDR or GDDR, so an analog to LRDIMMs is not a readily-available technology.

GDDR is the memory technology that operates at the edge of what's possible for per-pin bandwidth. Loading that memory bus down with many ranks is not something we can expect to be achievable by just putting down more pads on the PCB.

Re: Intel announces Arc B-series "Battlemage" discrete graphics with Linux support

#520

Post by stephen_cagle »

bayindirh wrote: post #264
treprinum wrote: post #26

Why don't they just release a basic GPU with 128GB RAM and eat NVidia's local generative AI lunch? The networking effect of all devs porting their LLMs etc. to that card would instantly put them as a major CUDA threat. But beancounters running the company would never get such an idea...

Disclosure: HPC admin who works with NIVIDA cards here. Because, no. It's not as simple as that. NVIDIA has a complete ecosystem now. They have cards. They have cards of cards (platforms), which they produce, validate and sell. They have NVLink crossbars and switches which connects these cards on their card of cards with very high speeds and low latency. For inter-server communication they have libraries which coordi…

Pretty sure they are talking about inference in the post you are responding to. Training the model obviously needs far more compute, but running them locally is what most devs are interested in.
Post reply on HN