111–120 of 135 posts
Waiting for Intel Battlemage, latest rumors says 16 GB of memory, a 256-bit wide bus, RTX 4080 level of performance in compute for 450$ MSRP. Wait and see Q3 2024. Intel software stack is more likely to challenge Nvidia's than AMD's.
Very much a "I'll believe it when I see it" scenario.
I thought it was weird that they would release new GPUs now, but I'm not complaining. The more GPUs the better.
I feel the exact opposite. There are too many! There are now nine models of the 4000-series, and that's not including laptop models or the cancelled 4080 12 GB.
IMO, there should be 5 models at the maximum. I shouldn't have to sit here and do a bunch of research to find out if the 4070 SUPER is faster than a 4070 Ti, and whether I should go for a 4070 SUPER Ti.
4060, 4070, 4080, 4090. That's all they ever needed. Budget, mid-grade, enthusiast, top-of-the-line. That's all that's needed.
Earlier quoted context omitted.
To me it's a bit strange because it assumes a buy / not buy perspective rather than a buy this / buy that perspective.
Welcome to PC hardware. People rarely switch camps or price tier. People who have 4070 today are the people who had 1070/2070/3070.
Earlier quoted context omitted.
Thanks! That rules them out completely EDIT: For training
Are you aware that cards containing “LLM” (40-80GB) levels of VRAM cost substantially more and the status quo for consumer cards hovers around 4-12GB, only going to 24GB for top end cards?
Give the consumers / gamers a consumer-priced GPU with a max of 16-24 GB VRAM for the high-end models. By consumer-priced, I mean $500-2000.
And make anyone interested in AI / ML / LLM / 3D / creatives pay $3000-10000 for GPUs that are similar in performance but have much higher VRAM.
Then top it out with six-figure (or higher) priced GPUs for the FAANG companies which can afford them for their data centers and currently contribute the most revenue (and profit) to NVidia.
Earlier quoted context omitted.
As the ancestor commenter mentioned: > If you’re interested in ML training Training - at least the one I tried - requires to be run in fp16 mode. So a 7b net needs 14 GB for the model weights alone, plus some extra for the context and the stuff I don't really understand (some gradient values, oh that makes sense now that I've written it)
I thought it was possible to use two cards and “share” the ram? Then it would still be a good deal. But maybe I’m wrong.
NVidia no longer supports this for the 40-series, I think this is because they want anyone interested in using their GPUs for LLMs to buy the pricier models with more VRAM.
Earlier quoted context omitted.
When you can sell an enterprise-grade card with 40-80GB of VRAM for $50k, selling consumer cards with 24GB for $2k is almost a form of charity, by comparison. AMD and Intel GPUs do not have the software ecosystem for AI workloads that Nvidia does, though AMD is rapidly improving. Nvidia has had an effective monopoly on the AI hardware space for the last year or so, and continues to have an effective near-monopoly, bu…
> When you can sell an enterprise-grade card with 40-80GB of VRAM for $50k ADA 6000 RTX (48GB) — an enterprise (workstation) cars — is about $10K sticker price. The differentiator between those and data center cards in the 40GB-80GB range is, obviously, not just VRAM.
The RTX ADA 6000 actually slightly performs worse than a consumer-grade 4090 (~10% less performant), even though it retails for 4-5x more ($8000 for ADA 6000 compared to $1500-2000 for a 4090).
I thought it was weird that they would release new GPUs now, but I'm not complaining. The more GPUs the better.
> The more GPUs the better. I feel the exact opposite. There are too many! There are now nine models of the 4000-series, and that's not including laptop models or the cancelled 4080 12 GB. IMO, there should be 5 models at the maximum. I shouldn't have to sit here and do a bunch of research to find out if the 4070 SUPER is faster than a 4070 Ti, and whether I should go for a 4070 SUPER Ti. 4060, 4070, 4080, 4090. That…
Though given that NVidia seems to release its consumer GPUs once every 2 years (20 series in late 2018, 30-series late 2020, 40 series in late 2022), I wonder if this is more of a marketing ploy - release the main series once every 2 years, but bring our "super" refreshers in the middle of the cycle to make sure you're still in the news, and get some segment of consumers / gamers to upgrade to those.