111–120 of 221 posts
Earlier quoted context omitted.
The problem is that the specs of AMD consumer-grade GPUs do not translate to computer performance when you try and chain more than one together. I have 7 NVidia 4090s under my desk happily chugging along on week long training runs. I once managed to get a Radeon VII to run for six hours without shitting itself.
Wow, are these 7 RTX 4090s in a single setup? Care to share more how you build it (case, cooling, power, ..)?
Earlier quoted context omitted.
Everything is comparative. AMD isn't perfect. As an Ex Shareholder I have argued they did well partly because of Intel's downfall. In terms of execution it is far from perfect. But Nvidia is a different beast. It is a bit like Apple in the late 00s where you take business, forecast, marketing, operation, software, hardware, sales etc You take any part of it and they are all industry leading. And having industry leadi…
Yeah, no. We are in the middle of a monopoly squeeze by NVidia on the most innovative part of the economy right now. I expect the DOJ to hit them harder than they did MS in the 90s given the bullshit they are pulling and the drag on the economy they are causing. By comparison if AMD could write a driver that didn't shit itself when it had to multiply more than two matrices in a row they'd be selling cards faster than…
Maybe the US will do something if GPU price becomes the limit instead of the supply of chips and power.
Earlier quoted context omitted.
The problem is that the specs of AMD consumer-grade GPUs do not translate to computer performance when you try and chain more than one together. I have 7 NVidia 4090s under my desk happily chugging along on week long training runs. I once managed to get a Radeon VII to run for six hours without shitting itself.
Wow, are these 7 RTX 4090s in a single setup? Care to share more how you build it (case, cooling, power, ..)?
Earlier quoted context omitted.
Peculiar business model, at a glance. It seems like they're doing work that AMD ought to be doing, and is probably doing behind the scenes. Who is the customer for a third-party GPU driver shim?
Is this effort benefiting everyone? I.e. where is it going / is it open source?
Earlier quoted context omitted.
They've actually been making positive moves with GPUs lately along with a success story for the B580.
B580 being a "success" is purely a business decision as a loss leader to get their name into the market. A larger die on a newer node than either Nvidia or AMD means their per-unit costs are higher, and are selling it at a lower price. That's not a long-term success strategy. Maybe good for getting your name in the conversation, but not sustainable.
Earlier quoted context omitted.
> If I can save someone else from wasting $100,000 on hardware and six months of their life then my post has done more good than the AMD marketing department ever will. This seems like unuseful advice if you've already given up on them. You tried it and at some point in the past it wasn't ready. But by not being ready they're losing money, so they have a direct incentive to fix it. Which would take a certain amount o…
I've tried it three times. I've seen people try it every six months for two decades now. At some point you just have to accept that AMD is not a serious company, but is a second rate copycat and there is no way to change that without firing everyone from middle management up. I'm deeply worried about stagnation in the CPU space now that they are top dog and Intel is dead in the water. Here's hoping China and Risk V s…
Have you tried compute shaders instead of that weird HPC-only stuff?
Compute shaders are widely used by millions of gamers every day. GPU vendors have huge incentive to make them reliable and efficient: modern game engines are using them for lots of thing, e.g. UE5 can even render triangle meshes with GPU compute instead of graphics (the tech is called nanite virtualized geometry). In practice they work fine on all GPUs, ML included: https://github.com/Const-me/Cgml
Earlier quoted context omitted.
The problem is that the specs of AMD consumer-grade GPUs do not translate to computer performance when you try and chain more than one together. I have 7 NVidia 4090s under my desk happily chugging along on week long training runs. I once managed to get a Radeon VII to run for six hours without shitting itself.
Wow, are these 7 RTX 4090s in a single setup? Care to share more how you build it (case, cooling, power, ..)?
There's a bunch of similar setups and there are a couple of dozen people that have done something similar on /r/localllama.
Earlier quoted context omitted.
Intel is in a vastly better shape than AMD, they have the software pretty much nailed down.
I've recently been poking around with Intel oneAPI and IPEX-LLM. While there are things that I find refreshing (like their ability to actually respond to bug reports in a timely manner, or at all) on a whole, support/maturity actually doesn't match the current state of ROCm. PyTorch requires it's own support kit separate from the oneAPI Toolkit (and runs slightly different versions of everything), the vLLM xpu suppor…
SYCL with llama.cpp is great though, at least at FP16 since it supports nothing else but even Arc iGPUs easily give 2-4x performance compared to CPU inference.
Intel should've just contributed to SYCL instead of trying to make their own thing and then forgot to keep maintaining it halfway through.