381–390 of 722 posts
Why don't they just release a basic GPU with 128GB RAM and eat NVidia's local generative AI lunch? The networking effect of all devs porting their LLMs etc. to that card would instantly put them as a major CUDA threat. But beancounters running the company would never get such an idea...
Disclosure: HPC admin who works with NIVIDA cards here. Because, no. It's not as simple as that. NVIDIA has a complete ecosystem now. They have cards. They have cards of cards (platforms), which they produce, validate and sell. They have NVLink crossbars and switches which connects these cards on their card of cards with very high speeds and low latency. For inter-server communication they have libraries which coordi…
Earlier quoted context omitted.
M3/M4 Max MacBooks with 128GB RAM are already way better than an A6000 for very large local LLMs. So even if the GPU is as slow as the one in M3/M4 Max (<3070), and using some basic RAM like LPDDR5x it would still be way faster than anything from NVidia.
The M4 Max needs an enormous 512bit memory bus to extract enough bandwidth out of those LPDDR5x chips, while the GPUs that Intel just launched are 192/160bit and even flagships rarely exceed 384bit. They can't just slap more memory on the board, they would need to dedicate significantly more silicon area to memory IO and drive up the cost of the part, assuming their architecture would even scale that wide without hit…
Earlier quoted context omitted.
I'm baffled that PC gamers have decided that 1440p is the endgame for graphics. When I look at a 27-inch 1440p display, I see pixel edges everywhere. It's right at the edge of losing the visibility of individual pixels, since I can't perceive them at 27-inch 2160p, but not quite there yet for desktop distances. Time marches on, and I become ever more separated from gaming PC enthusiasts.
This is a trade-off with frame rates and rendering quality. When having to choose, most gamers prefer higher frame rate and rendering quality. With 4K, that becomes very expensive, if not impossible. 4K is 2.25 times the pixels of 1440p, which for example means you can get double the frame rate with 1440p using the same processing power and bandwidth. In other words, the current tech just isn’t quite there yet, or no…
It’s a great way to have my cake and eat it too.
Earlier quoted context omitted.
I think plenty of enthusiastic open source devs would jump at it and fix their software if the software was reasonably open. The same effect as what happened when Meta released LLaMA.
It is open and they regularly merge PRs. https://github.com/ROCm/ROCm/pulls?q=is%3Apr+is%3Aclosed
Earlier quoted context omitted.
No you don‘t need much bandwidth between cards for inference
Copying daemons (gdrcopy) is about pumping data in and out of a single card. docker-nvidia and rest of the stack is enablement for using cards. GPU-Direct is about pumping data from storage devices to cards, esp. from high speed storage systems across networks. MIG actually shares a single card to multiple instances, so many processes or VMs can use a single card for smaller tasks. Nothing I have written in my previo…
Earlier quoted context omitted.
ML is about hit another winter. Maybe intel is ahead of industry. Or we can keep asking high computers questions about programming.
> ML is about hit another winter. I agree ML is about to hit (or has likely already hit) some serious constraints compared to breathless predictions of two years ago. I don't think there's anything equivalent to the AI winter on the horizon, though—LLMs even operated by people who have no clue how the underlying mechanism functions are still far more empowered than anything like the primitives of the 80s enabled.
I'm still not convinced about that. All the """studies""" show 30-60% boost in productivity but clearly this doesn't translate to anything meaningful in real life because no industry laid off 30-60% of their workforce and no industry progressed anywhere close to 30% since chat gpt was released.
It's been released a whole 24 months ago, remember the talks about freeing us from work and curing cancer... Even investments funds which are the biggest suckers for anything profitable are more and more doubtful
Earlier quoted context omitted.
Gaming at 2160p is just too expensive still, imo. You gotta pay more for your monitor, GPU and PSU. Then if you want side monitors that match in resolution, you're paying more for those as well. You say PC gamers at the start of your comment and gaming PC enthusiasts at the end. These groups are not the same and I'd say the latter is largely doing ultrawide, 4k monitor or even 4k TV. According to steam, 56% are on 10…
I'm still using a 50" 1080p (plasma!) television in my living room. It's close to 15 years old now. I've seen newer and bigger TVs many times at my friends house, but it's just not better enough that I can be bothered to upgrade.
I think a graphics card tailored for 2k gaming is actually great. 2k really is the goldilocks zone between 4k and 1080p graphics before you start creeping into diminishing returns.
Can it compete with the massive used GPU market though? Why buy a new Intel card when I can get a used Nvidia card that I know will work well?
Earlier quoted context omitted.
While I'd agree monetisation seems to be a challenge in the long term (analogy: spreadsheets are used everywhere, but are so easy to make they're not themselves a revenue stream, only as part of a bigger package)… > Nvidia had a revenue of $27billion in 2023 - that's about $160 per person per year [0] for every working age person in the USA As a non-American, I'd like to point out we also earn money. > as no AI is go…
Sure, robotics help many jobs, and some level of the current deep learning boom seems to have crossover in improving that - but how many of them are running LLMs that affect Nvidia's bottom line right now? There's some interesting research in that area, but it's certainly not the primary driving force. And then is the control system the limiting factor for many systems - it's probably relatively easy to get a machine…
Nonetheless, Starbucks does not use these machines, and I don't see any reason that AI, on its current trajectory, will change that calculation any time soon.