71–80 of 464 posts
I like Framework and own one of their laptops. But the desktop seems more a triumph of gimmicky marketing than a desktop that's meaningfully different. And, it seems significantly overpriced.
How is AMD GPU compatibility with leading generative AI workflows? I'm under the impression everything is CUDA.
llama.cpp combined with Mesa’s Vulkan support for AMD GPUs has worked pretty well with everything I’ve thrown it at.
> But when we switch to longer context, we see something interesting happen. WMMA + FA basically loses no performance at this longer context length!
> Vulkan + FA still has better pp but tg is significantly lower. More data points would be better, but seems like Vulkan performance may continue to decrease as context extends while the HIP+rocWMMA backend should perform better.
lhl has also been sharing these test results in https://forum.level1techs.com/t/strix-halo-ryzen-ai-max-395-..., and his latest comment provides a great summary of the current state:
> (What is bad is that basically every single model has a different optimal backend, and most of them have different optimal backends for pp (handling context) vs tg (new text)).
Anyway, for me, the greatest thing about the Strix Halo + llama.cpp combo is that we can throw one or more egpu into the mix, as echoed by level1tech video (https://youtu.be/ziZDzrDI7AM?t=485), which should help a lot with PP performance.
Earlier quoted context omitted.
The 128GB version wouldn't be throwaway, since that's the maximum the platform as a whole supports anyway- more memory than that would require a new mainboard and CPU at the same time.
Not for upgrade reasons but what if you have a fault in one of the dimms or etc? Now you can't just drop in a replacement without changing everything.
Without that, it's really not a interesting solution.
demanding replaceable ram means also not wanting the benefits of the integrated memory
Earlier quoted context omitted.
He is still right. It is a desktop PC that is less repairable than all other desktop PCs, from a brand that is known to champion repairability. They had a reason for it, but could've chosen to not create more throwaway things.
I have been continuously baffled by the people that think that soldered on RAM is somehow "throwaway". My last desktop build is eight years old and I have never upgraded the ram. Never will. My next build will have an entirely new motherboard, ram, and GPU, and the last set will end up at the ewaste recycler, because who could I find that wants that old hardware? Soldered RAM, CPU, and GPU, that give space benefits a…
One of the primary objections to soldered RAM was/is the cost to purchase. As the likes of Apple priced Ram upgrade at a hefty premium to retail prices.
> In some ways, the Framework Desktop is a curious machine. Desktop PCs are already very user-repairable! So why is Framework even bringing their talents to this domain? In the laptop realm, they're basically alone with that concept, but in the desktop space, it's rather crowded already. Yet it somehow still makes sense. And even more curious, Framework Desktop is deliberately less repairable than their laptops. They…
According to the Framework CEO on the Linus Tech Tips video about this thing [1], they tried and AMD assigned an engineer on getting modular memory to work and decided it's not possible. Unless there's another company out there shipping this CPU with replaceable memory, I'll believe them. Even with LPCAMM, physics just doesn't work out. Of course, there are plenty of slower desktops you can buy where all the memory c…
very impressive... Max+ 395 specced with: 128GB of non-upgradeable LPDDR5x WD_BLACK SN850X NVMe - M.2 2280 - 8TB Noctua fan 3x + 3x extra USB-A & USB-C ports No OS option. only $2,776.00!!!
Bear in mind the gpu has access to all of that 128GB as well, so for AI thats very very cheap.
People seem to really not understand the limits of wanting unified memory architecture.
Earlier quoted context omitted.
The RAM is soldered on all Halo Strix platforms because physics is getting in the way. With pluggable DIMMs the memory bandwidth would be halved, at best.
Why not make a platform with a greater number of channels
For example Nvidia seek to ban consumer GPU use in datacenters as they to sell datacentre GPUs.
If they made consumer platforms that can take 1tb of ram etc, then people may choose to not buy EYPC.
Afterall many cloud providers already offer Ryzen VPS's.
Earlier quoted context omitted.
I have been continuously baffled by the people that think that soldered on RAM is somehow "throwaway". My last desktop build is eight years old and I have never upgraded the ram. Never will. My next build will have an entirely new motherboard, ram, and GPU, and the last set will end up at the ewaste recycler, because who could I find that wants that old hardware? Soldered RAM, CPU, and GPU, that give space benefits a…
> Not everybody upgrades RAM, and those people deserve computers too. No. It's end of the line with consumerism and we either start repairing and recycling or we die. Framework catered to people who agree with that, and this product is not in line. I have no idea why you would not upgrade your memory, I have done so in all PCs I ever owned and all laptops, and it's a very common (and cheap) upgrade. It reduces waste…
Upgrading the RAM would have created more waste than properly sizing the RAM to COU proportion from the beginning.
It is very odd to encounter someone who has such a narrow view of computing that they cannot imagine someone not upgrading their RAM.
I have not once, literally not once have RAM break either. I have been part of the management of clusters of hundreds of compute nodes, that would occasionally each have their failures, but not once was RAM the cause of failure. I'm fairly shocked to hear that anybody's RAM has failed, honestly, unless it's been overlocked or something else.
Earlier quoted context omitted.
Sockets have resistance and crosstalk, which affects signal integrity.
Wait, your telling me, I should have been desoldering the sockets off my motherboard, and directly soldering my RAM to the leads this entire time?
Earlier quoted context omitted.
That's a question for AMD and TCMC. They only have so much space on the silica. More memory channels means less of something else. This is not a "framework platform" issue, it's the specification of that CPU.
Well they chose to use this hardware platform. It all sounds like market segmentation to me, now that AMD is on top.