Earlier quoted context omitted.
As the ancestor commenter mentioned: > If you’re interested in ML training Training - at least the one I tried - requires to be run in fp16 mode. So a 7b net needs 14 GB for the model weights alone, plus some extra for the context and the stuff I don't really understand (some gradient values, oh that makes sense now that I've written it)
I thought it was possible to use two cards and “share” the ram? Then it would still be a good deal. But maybe I’m wrong.
New GeForce RTX 4080 Super, RTX 4070 Ti Super, RTX 4070 Super
121–130 of 135 posts
Re: New GeForce RTX 4080 Super, RTX 4070 Ti Super, RTX 4070 Super
#122Earlier quoted context omitted.
With the current generation of text and image generators, 24GB is the sweet spot
I wouldn't be shocked if the 5090 is also the only one with 24GB. Seems like NVidia is trying their hardest to suppress memory increases.
Re: New GeForce RTX 4080 Super, RTX 4070 Ti Super, RTX 4070 Super
#123Earlier quoted context omitted.
This is another very good release that is also being somewhat bizarrely panned too. Yeah, it's essentially a 7600 16GB/4060 16GB, and that's what people wanted a few months ago, more VRAM. It also opens up a bunch of possibilities for cheap ML (in the same way the 4060 Ti 16GB does). IMO this pretty well displaces the 4060 8GB and 16GB - it's cheaper (than even the already below-MSRP street prices) on 4060 Ti 8GB, it…
It's tough because most of the games being benchmarked by reviewers were designed to work well on 4GB for 1080p and on 8GB high settings. So the extra VRAM doesn't help much for benchmarking. But future games are likely to run better in >8GB simply because the PS5 and XBox Series X have more than 8.
I think 8GB is going to continue to be a long-lived target especially at 1080p resolutions (with whatever gains can be squeezed from upscaling etc too - although generally DLSS needs to inference against the full-quality textures etc). Series S has 8GB (of fast-partition ram, the rest is GTX 970 style slow-partition) and even Series X only has 10GB.
People also aren't giving enough credit to mesh shaders etc, the GTX 1650 is actually still in the game with 4GB in Alan Wake 2[0], it does make a difference. The "but a 2060 super isn't relevant anymore!" argument relies on the assumption that you're deciding not to turn on upscaling etc. 1650 can run AW2 on lowest-settings 1080p with 4gb with FSR2 and it looks fine, and it'd be even better with RTX/DLSS. 2060 with DLSS can do a console-like experience on AW2 zero problem.
[0] https://www.youtube.com/watch?v=vFf8NsOi-HU&t=356s
Consoles have always been a mixed bag. Yeah, they get a lot of specific tweaking and they also have special hardware which helps somewhat. But overall you're working from a (later in the gen) fairly low baseline. 6700 non-XT performance is ok but nothing stellar, and optimization doesn't save that. But honestly what they are good at is removing "paralysis of choice", having too many choices really hurts people and the emotional feeling of having to turn the setting onto lowest hurts people, even if that's what the console does itself! It's at least pre-tweaked lowest settings etc (although often worse tech etc - FSR3 is blown away by DLSS 3.5 image quality let alone 4.0 and future iterations which aren't far away). You don't have to think about it, you just say "framerate or quality" and you probably know which you want.
That is the problem that people will struggle with. 8GB will still work. It just also will be a Series S level experience, modulo things like mesh shading that occasionally differentiate the consoles (PS5 lacks it iirc, as well as DP4a). And that can still often look fine. Will you get more from spending more? Yes. But it also doesn't take that much - series X is the 6700, series S is like APU territory. A 3080 blows away the series X, etc. But you will have to stomach through moving that slider from "native" to "performance" and the texture quality from "ultra" to "medium". Etc. People have lost touch of the world of yesterday when "can you run crysis" was an actual question and not a meme, slamming every setting to ultra is not a given when you buy an entry-level card, and people also can't handle the fact that $200-300 is now entry-level. Midrange is $500-700, high-end is $800 to "how much have you got".
And that's not NVIDIA, that's really just wafer costs. If you want to compare die sizes and MSRPs against 10+ years ago (look up GTX 670/GK104 lol), you have to bear in mind that a given die size might cost 5x what it did back then. And it increases ~30% every node-family since 28nm, more or less. It's gonna go up over time, if you aren't moving up in price you're moving down in product-design-bracket and are going to have to deal with more design compromises to hit those lower price-points in the face of rising costs. It sucks, but nobody has any better ideas - to paraphrase what someone once told me, "the industrial and creative poles of several societies and continents are laser-focused on pushing this backwards, and yet the problems only become more difficult after each success". There is no easy answer, lots of smart people are working at this.
Re: New GeForce RTX 4080 Super, RTX 4070 Ti Super, RTX 4070 Super
#124Earlier quoted context omitted.
This was what I came to comment. Very strange. This article https://beebom.com/nvidia-rtx-4080-4070-ti-super-gpu-specs-r... says "the RTX 4070 Ti Super is up to 10% faster than the non-Super 4070 Ti on average" So, on average, up to 10% faster, yeah seems pretty incremental.
The big difference with the 4070 Ti Super is that it's using the AD103 chip (with a full 256-bit memory bus and 16GB of VRAM) found in the 4080, which is a huge leap over the AD104 chip found in the 4070 Ti (non-Super), which only touts a 192-bit memory bus and 12GB of VRAM. While the TFLOPS of the Super variant does only see a ~10% increase as you note, memory bandwidth jumps by 42% and the memory capacity jumps by…
I got a 4090 a few months ago before the prices increased, and I'm beyond stoked with the performance for (typically triple qhd simulation) gaming. It's just a beast.
I have a 2nd PC I'd like to upgrade too though, and the 4070 TI looks like it would be fantastic in this.
Re: New GeForce RTX 4080 Super, RTX 4070 Ti Super, RTX 4070 Super
#125Earlier quoted context omitted.
It's not always faster than the 3090, let alone "significantly", except in very specific resolutions and games. here's a review from a traditionally pro intel and nvidia outlet: https://www.techpowerup.com/review/nvidia-geforce-rtx-4080-f...
I checked a sampling of a few games there and the only one where it's not ahead by a solid margin at 2560x1440 is Far Cry 6... the game that I gave up trying to play because it would not run smoothly on my 3090 + 5900X even with the settings turned down.
Or one of us sees what they want to see.
Re: New GeForce RTX 4080 Super, RTX 4070 Ti Super, RTX 4070 Super
#126Earlier quoted context omitted.
When you can sell an enterprise-grade card with 40-80GB of VRAM for $50k, selling consumer cards with 24GB for $2k is almost a form of charity, by comparison. AMD and Intel GPUs do not have the software ecosystem for AI workloads that Nvidia does, though AMD is rapidly improving. Nvidia has had an effective monopoly on the AI hardware space for the last year or so, and continues to have an effective near-monopoly, bu…
> When you can sell an enterprise-grade card with 40-80GB of VRAM for $50k ADA 6000 RTX (48GB) — an enterprise (workstation) cars — is about $10K sticker price. The differentiator between those and data center cards in the 40GB-80GB range is, obviously, not just VRAM.
Also, as samspenc points out, that RTX 6000 is using the same AD102 chip found in a consumer RTX 4090, just with marginally more CUDA cores, TMUs, ROPs, etc.
The most substantial difference between the two is that the $10k card has twice the VRAM of the $2k card.
I know it sounds outrageously oversimplified, but Nvidia can indeed more or less print money by attaching a few extra memory chips to what is otherwise a flagship consumer-oriented graphics processor.
The profit margins on the H100, for reference, are estimated to be around one thousand percent (1000%), i.e., they sell for ~10x as much as it costs Nvidia to make them, and demand for them still grossly exceeds the total supply. See: https://the-decoder.com/nvidias-h100-gpu-sells-like-hot-cake...
Re: New GeForce RTX 4080 Super, RTX 4070 Ti Super, RTX 4070 Super
#127Re: New GeForce RTX 4080 Super, RTX 4070 Ti Super, RTX 4070 Super
#128Earlier quoted context omitted.
Appreciate the extra insight here! I hope for the sake of purchasers it is only 12% cost increase, but I have a suspicion if there's more than 12% extra value, we'll see it in the price
Just checked the CES announcement and updated my post to reflect that it actually has the same launch price that the 4070 Ti (non-Super) had! Amazing bargain!
Re: New GeForce RTX 4080 Super, RTX 4070 Ti Super, RTX 4070 Super
#129Earlier quoted context omitted.
Yeah I'm doing totally fine with an i7-9700K, although having 32gb sure helps. 2060Super GPU is starting to feel dated though!
My own 9700k is starting to creak a bit, but that was my own mistake buying a 144hz monitor.
Re: New GeForce RTX 4080 Super, RTX 4070 Ti Super, RTX 4070 Super
#130Earlier quoted context omitted.
Welcome to PC hardware. People rarely switch camps or price tier. People who have 4070 today are the people who had 1070/2070/3070.
Rather 1080/2080/3080 with current prices.
Not everyone is buying the best and latest. Plenty of people wait for previous gens to drop in prices or enter the secondary market.