371–380 of 722 posts
Why don't they just release a basic GPU with 128GB RAM and eat NVidia's local generative AI lunch? The networking effect of all devs porting their LLMs etc. to that card would instantly put them as a major CUDA threat. But beancounters running the company would never get such an idea...
Intel can't compete head to head with Nvidia on performance. But surely it's easy enough to compete on video ram - why not load their GPUs to the max with video ram? And also video encoder cores - Intel has a great video encoder core and these vary little across high end to low end GPUs - so they could make it a standout feature to have, for example, 8 video encoder cores instead of 2. It's no wonder Nvidia is the ki…
Why don't they just release a basic GPU with 128GB RAM and eat NVidia's local generative AI lunch? The networking effect of all devs porting their LLMs etc. to that card would instantly put them as a major CUDA threat. But beancounters running the company would never get such an idea...
Disclosure: HPC admin who works with NIVIDA cards here. Because, no. It's not as simple as that. NVIDIA has a complete ecosystem now. They have cards. They have cards of cards (platforms), which they produce, validate and sell. They have NVLink crossbars and switches which connects these cards on their card of cards with very high speeds and low latency. For inter-server communication they have libraries which coordi…
Start by being a "second vendor" for huge customers of NVIDIA that want to foster competition, as well as a few others willing to take risks, and build from there.
Who is the target audience for this? Well informed gamers know Intel's discrete GPU is hanging by a thread, so they're not hoping on that bandwagon. Too small for ML. The only people really happy seem to be the ones buying it for transcoding and I can't imagine there is a huge market of people going "I need to go buy a card for AV1 encoding".
Ubuntu 24.04 couldn't even boot to a tty with the Nvidia Quadro thing that came with this major-brand PC workstation, still under warranty.
Earlier quoted context omitted.
My comment was about Intel having a starter project, getting enthusiastic response from devs, network effects and iterate from there. They need a way to threaten Nvidia and just focusing on what they can't do won't bring them there. There is one route where they can disturb Nvidia's high end over time and that's a cheap basic GPU with lots of RAM. Like Ryzen 1st gen whose single core performance was two generations b…
It would be a good idea to start with some basic understanding of GPU, and realizing why this can't easily be done.
Earlier quoted context omitted.
4090 is 5x faster than M3 Max 128GB according to my tests but it can't even inference LLaMA-30B. The moment you hit that memory limit the inference is suddenly 30x slower than M3 Max. So a basic GPU with 128GB RAM would trash 4090 on those larger LLMs.
Do you have the code for that test?
Who is the target audience for this? Well informed gamers know Intel's discrete GPU is hanging by a thread, so they're not hoping on that bandwagon. Too small for ML. The only people really happy seem to be the ones buying it for transcoding and I can't imagine there is a huge market of people going "I need to go buy a card for AV1 encoding".
Why would that matter? You buy one GPU, in a few years you buy another GPU. It's not a life decision.
Earlier quoted context omitted.
For sure its been a sweet spot for a very long time for budget conscious gamers looking for best balance of price and frame rates, but 1440p optimized parts are nothing new. Both NVidia and AMD make parts that target 1440p display users too, and have done for years. Even previous Intel parts you can argue were tailored for 1080p/1440p use, given their comparative performance deficit at 4k etc. Assuming they retail at…
I'm baffled that PC gamers have decided that 1440p is the endgame for graphics. When I look at a 27-inch 1440p display, I see pixel edges everywhere. It's right at the edge of losing the visibility of individual pixels, since I can't perceive them at 27-inch 2160p, but not quite there yet for desktop distances. Time marches on, and I become ever more separated from gaming PC enthusiasts.
Tbh now that I think about it I only really need resolution for general usage. For gaming I'm running everything but textures on low with min or max FOV depending on the game so it's not exactly aesthetic anyway. I more so need physical screen size so the heads are physically larger without shoving my face in it and refresh rate.
I think a graphics card tailored for 2k gaming is actually great. 2k really is the goldilocks zone between 4k and 1080p graphics before you start creeping into diminishing returns.
For sure its been a sweet spot for a very long time for budget conscious gamers looking for best balance of price and frame rates, but 1440p optimized parts are nothing new. Both NVidia and AMD make parts that target 1440p display users too, and have done for years. Even previous Intel parts you can argue were tailored for 1080p/1440p use, given their comparative performance deficit at 4k etc. Assuming they retail at…
Earlier quoted context omitted.
What if they put 8 identical GPUs in the package, each with 1/8 the memory? Would that be a useful configuration for a modern LLM?
GPU inference is always a balancing act, trying to avoid bottlenecks on memory bandwidth (loading data from the GPU's global memory/VRAM to the much smaller internal shared memory, where it can be used for calculations) and compute (once the values are loaded). Splitting the model up between several GPUs would add a third much worse bottleneck – memory bandwidth between the GPUs. No matter how well you connect them,…
What if you allowed the system to only have a shared memory between every neighboring pair of GPUs?
Would that make sense for an LLM?