191–200 of 722 posts
12GB max is a non-starter for ML work now. Why not come out with a reasonably priced 24gb card even if it isn't the fastest and target it at the ML dev world? Am I missing something here?
ML is about hit another winter. Maybe intel is ahead of industry. Or we can keep asking high computers questions about programming.
Earlier quoted context omitted.
For sure its been a sweet spot for a very long time for budget conscious gamers looking for best balance of price and frame rates, but 1440p optimized parts are nothing new. Both NVidia and AMD make parts that target 1440p display users too, and have done for years. Even previous Intel parts you can argue were tailored for 1080p/1440p use, given their comparative performance deficit at 4k etc. Assuming they retail at…
I'm baffled that PC gamers have decided that 1440p is the endgame for graphics. When I look at a 27-inch 1440p display, I see pixel edges everywhere. It's right at the edge of losing the visibility of individual pixels, since I can't perceive them at 27-inch 2160p, but not quite there yet for desktop distances. Time marches on, and I become ever more separated from gaming PC enthusiasts.
You say PC gamers at the start of your comment and gaming PC enthusiasts at the end. These groups are not the same and I'd say the latter is largely doing ultrawide, 4k monitor or even 4k TV.
According to steam, 56% are on 1080p, 20% on 1440p and 4% on 2160p.
So gamers as a whole are still settled on 1080p, actually. Not everyone is rich.
Earlier quoted context omitted.
What makes local AI interesting to you vs larger remote models like ChatGPT and Claude?
Not OP but for me a big thing is privacy, I can feed it personal documents and expect those to not leak. It has zero cost, hardware is already there. I'm not captive to some remote company. I can fiddle and integrate with other home sensors / automation as I want.
Earlier quoted context omitted.
They can use LPDDR5x, it would still massively accelerate inference of large local LLMs that need more than 48GB RAM. Any tensor swapping between CPU RAM and GPU RAM kills the performance.
I think we don't really disagree, I just think that this shape isn't really a gpu its just a cpu because it isn't very good for graphics at that point.
Earlier quoted context omitted.
Launching a new SKU for $500-1000 with 48gb of RAM seems like a profitable idea. The GPU isn't top-of-the-line, but the RAM would be unmatched for running a lot of models locally.
You can’t just throw in more RAM without having the rest of the GPU architected for it. So there’s an R&D cost involved for such a design, and there may even be trade-offs on performance for the mass-market lower-tier models. I’m doubtful that the LLM enthusiast/tinkerer market is large enough for that to be obviously profitable.
Honestly, Apple seems to be on the right track here. DDR5 is slower than GDDR6, but you can scale the amount of RAM far higher simply by swapping out the density.
Earlier quoted context omitted.
> ML is about hit another winter. I agree ML is about to hit (or has likely already hit) some serious constraints compared to breathless predictions of two years ago. I don't think there's anything equivalent to the AI winter on the horizon, though—LLMs even operated by people who have no clue how the underlying mechanism functions are still far more empowered than anything like the primitives of the 80s enabled.
I think there'll be a "financial" winter - or another way a bubble burst - the investment right now is simply unsustainable, how are these products going to be monetized? Nvidia had a revenue of $27billion in 2023 - that's about $160 per person per year [0] for every working age person in the USA. And it's predicted to more than double in 2024. If you reduce that to office workers (you know, the people who might actu…
Earlier quoted context omitted.
> ML is about hit another winter. I agree ML is about to hit (or has likely already hit) some serious constraints compared to breathless predictions of two years ago. I don't think there's anything equivalent to the AI winter on the horizon, though—LLMs even operated by people who have no clue how the underlying mechanism functions are still far more empowered than anything like the primitives of the 80s enabled.
I think there'll be a "financial" winter - or another way a bubble burst - the investment right now is simply unsustainable, how are these products going to be monetized? Nvidia had a revenue of $27billion in 2023 - that's about $160 per person per year [0] for every working age person in the USA. And it's predicted to more than double in 2024. If you reduce that to office workers (you know, the people who might actu…
12GB max is a non-starter for ML work now. Why not come out with a reasonably priced 24gb card even if it isn't the fastest and target it at the ML dev world? Am I missing something here?
This is a graphics card.
Earlier quoted context omitted.
Launching a new SKU for $500-1000 with 48gb of RAM seems like a profitable idea. The GPU isn't top-of-the-line, but the RAM would be unmatched for running a lot of models locally.
You can’t just throw in more RAM without having the rest of the GPU architected for it. So there’s an R&D cost involved for such a design, and there may even be trade-offs on performance for the mass-market lower-tier models. I’m doubtful that the LLM enthusiast/tinkerer market is large enough for that to be obviously profitable.
People did it with the RTX3070. https://www.tomshardware.com/news/3070-16gb-mod