71–80 of 98 posts
Earlier quoted context omitted.
Based on leaks, it looks like intel somehow missed an easy opportunity here. There is an insane demand for high VRAM cards now, and it seems the next intel cards will be 12GB. Intel, screw everything else, just pack as much VRAM in those as you can. Build it and they will come.
Exactly, I'd love to have 1TB of RAM that can be accessed at 6000 MT/s.
Earlier quoted context omitted.
I think the answer to that is fairly straightforward. Intel isn't in the business of producing RAM. They would have to buy and integrate a third-party product which is likely not something their business side has ever contemplated as a viable strategy.
Their GPUs as sold already include RAM.
Earlier quoted context omitted.
No kidding... Intel is playing catch-up with Nvidia in the AI space and a big reason for that is their offerings aren't competitive. You can get an Intel Arc A770 with 16GB of VRAM (which was released in October, 2022) for about $300 or an Nvidia 4060 Ti with 16GB of VRAM for ~$500 which is twice as fast for AI workloads in reality (see: https://cdn.mos.cms.futurecdn.net/FtXkrY6AD8YypMiHrZuy4K-120... ) This is a huge…
I don't agree. Who will buy it? A few enthusiasts who wants to run LLM locally but cannot afford M3 or 4090? It will be a niche product with poor sales.
Earlier quoted context omitted.
But even if Intel have stable optimized drivers and ML support, it'd still need to be supported by PyTorch/etc for most developers to want to use it. People want to write at high level, not at CUDA-type level.
Intel is supported in Pytorch, though. It's supported from their own branch, which is presumably a big annoyance to install, but they do work
Earlier quoted context omitted.
Their GPUs as sold already include RAM.
Yes, but they don't fab their own RAM. It's a cost center for them.
Earlier quoted context omitted.
It doesn't matter whether anyone is "spoiled" or not. The fact is large language models require a lot of VRAM, and the more interesting ones need more than 24GB to run. The people who are able to afford systems with more than 24GB VRAM will go buy hardware that gives them that, and when GPU vendors release products with insufficient VRAM they limit their market. I mean inequality is definitely increasing at a worryin…
I'm just fascinated that the response/demand to running out of RAM is "Just sell us more RAM, god damn!" instead of engineering a solution to make due with what is practically (and realistically) available.
i learned my RAM lesson when I bought my first real linux PC. it had 4MB of RAM, which was enough to run X, bash, xterm, and emacs. But once I ran all that and also wanted to compile with g++, it would start swapping, which in the days of slow hard drives, was death to productivity.
I spent $200 to double to 8MB, and then another $200 to double to 16MB, and then finally, $200 to max out the RAM on my machine-- 32MB! And once I did that everything flew.
Rather than attempting to solve the problem by making emacs (eight megs and constantly swapping) use less RAM, or find a way to hack without X, I deployed money to max out my machine (which was practical, but not realistically available to me unless I gave up other things in life for the short term). Not only was I more productive, I used that time to work on other engineering problems which helped build my career, while also learning an important lesson about swapping/paging.
People demand RAM and what was not practically available is often available 2 years later as standard. Seems like a great approach to me, especially if you don't have enough smart engineers to work around problems like that (see "How would you sort 4M integers in 2M of RAM?")
Earlier quoted context omitted.
It doesn't matter whether anyone is "spoiled" or not. The fact is large language models require a lot of VRAM, and the more interesting ones need more than 24GB to run. The people who are able to afford systems with more than 24GB VRAM will go buy hardware that gives them that, and when GPU vendors release products with insufficient VRAM they limit their market. I mean inequality is definitely increasing at a worryin…
I'm just fascinated that the response/demand to running out of RAM is "Just sell us more RAM, god damn!" instead of engineering a solution to make due with what is practically (and realistically) available.
Earlier quoted context omitted.
I don't agree. Who will buy it? A few enthusiasts who wants to run LLM locally but cannot afford M3 or 4090? It will be a niche product with poor sales.
AFAIK, unless you are a huge American corp with orders above $100m Nvidia will only sell you old and expensive server cards like the crappy A40 PCIe 4.0 48GB GDDR6 at $5,000. Good luck getting SXM H100s or GH200. If Intel sells a stackable kit with a lot of RAM and a reasonable interconnect a lot of corporate customers will buy. It doesn't even have to be that good, just half way between PCIe 5.0 and NVLink. But it s…
However, server solutions could have some traction.
Earlier quoted context omitted.
It doesn't matter whether anyone is "spoiled" or not. The fact is large language models require a lot of VRAM, and the more interesting ones need more than 24GB to run. The people who are able to afford systems with more than 24GB VRAM will go buy hardware that gives them that, and when GPU vendors release products with insufficient VRAM they limit their market. I mean inequality is definitely increasing at a worryin…
I'm just fascinated that the response/demand to running out of RAM is "Just sell us more RAM, god damn!" instead of engineering a solution to make due with what is practically (and realistically) available.
The people training 70B parameter models from scratch need ~600GB of VRAM to do it!