101–110 of 464 posts
Earlier quoted context omitted.
For LLM inference, I don't think the PCIe bandwidth matters much and a GPU could improve greatly the prompt processing speed.
Only if your entire model fits the GPU VRAM. To me this reads like "if you can afford those 256GB VRAM GPUs, you don't need PCIe bandwidth!"
Earlier quoted context omitted.
Every single description of the Framework desktop that I've seen has addressed this issue. To comment as though it's some sort of mystery is disingenuous at best. My comment was precisely as friendly as the commenter deserved. And as I said, if you read the article, you'll see that the tradeoff in question has paid off very well.
You completely missed the point of my original comment, I'll take a second stab at it: 1. Framework branded themselves as the company for DIY computer repairability and maintainability in the laptop space. 2. They've now released a desktop that is less repairable than their laptops, and much less repairable than most desktops you can buy today. That's what I consider a curious move. The hardware choice may provide a…
I don't see what you see. It's a single product, not a realignment of their business model. They saw an opportunity and brought to market a product that will likely sell out, which tells us that customers are happy to make the trade-off of modularity and repairability for what the Strix Halo brings to the table. I think your interpretation of their mission is a bit uncharitable, maybe naive, and leaves the company little room to be a company.
How is AMD GPU compatibility with leading generative AI workflows? I'm under the impression everything is CUDA.
CUDA isn't really used for new code. Its used for legacy codebases. In the LLM world, you really only see CUDA being used with Triton and/or PyTorch consumers that haven't moved onto better pastures (mainly because they only know Python and aren't actually programmers). That said, AMD can run most CUDA code through ROCm, and AMD officially supports Triton and PyTorch, so even the academics have a way out of Nvidia he…
How is AMD GPU compatibility with leading generative AI workflows? I'm under the impression everything is CUDA.
CUDA isn't really used for new code. Its used for legacy codebases. In the LLM world, you really only see CUDA being used with Triton and/or PyTorch consumers that haven't moved onto better pastures (mainly because they only know Python and aren't actually programmers). That said, AMD can run most CUDA code through ROCm, and AMD officially supports Triton and PyTorch, so even the academics have a way out of Nvidia he…
How is AMD GPU compatibility with leading generative AI workflows? I'm under the impression everything is CUDA.
CUDA isn't really used for new code. Its used for legacy codebases. In the LLM world, you really only see CUDA being used with Triton and/or PyTorch consumers that haven't moved onto better pastures (mainly because they only know Python and aren't actually programmers). That said, AMD can run most CUDA code through ROCm, and AMD officially supports Triton and PyTorch, so even the academics have a way out of Nvidia he…
Reads like an advertisement to me.
I doubt that Dhh if all people would do advertisement (as paid) for this. He genuinely seems to enjoy his macos break
I like Framework and own one of their laptops. But the desktop seems more a triumph of gimmicky marketing than a desktop that's meaningfully different. And, it seems significantly overpriced.
How is AMD GPU compatibility with leading generative AI workflows? I'm under the impression everything is CUDA.
CUDA isn't really used for new code. Its used for legacy codebases. In the LLM world, you really only see CUDA being used with Triton and/or PyTorch consumers that haven't moved onto better pastures (mainly because they only know Python and aren't actually programmers). That said, AMD can run most CUDA code through ROCm, and AMD officially supports Triton and PyTorch, so even the academics have a way out of Nvidia he…