A beginner's guide to deploying LLMs with AMD on Windows using PyTorch
1–10 of 48 posts
Re: A beginner's guide to deploying LLMs with AMD on Windows using PyTorch
#2I have a philosophy for which I have mixed feelings because I like it in principle despite it making me worse off in some other ways:
Devs should punish companies that clearly don't give a shit about them.
AMD's response? Nothing. They do build AI chips but they're for the megacorps. At one point I wanted to save for a MI300x. But AMD only sells them in batches of x8! (not factorial lol).
I've had it with AMD, and would rather penalize them (even if it costs them pennies) out of principle.
I had similar thoughts about Apple MLX a while back and wrote a mean post about it on reddit: https://www.reddit.com/r/LocalLLaMA/comments/19cdd9z/dont_ta...
But since then the cracked MLX team has kept delivering so much that I've now completely changed my mind about it. That, coupled with Apple's failed AI attempts and the recent addition of MatMul in A-chips (and ofc in M5), gives me hope about MLX's future and now I develop for it!
Re: A beginner's guide to deploying LLMs with AMD on Windows using PyTorch
#3I have a philosophy for which I have mixed feelings because I like it in principle despite it making me worse off in some other ways: Devs should punish companies that clearly don't give a shit about them. When I see AMD, I think of a firm that heavily prioritized their B2B business over B2C. Not just gamers, but a lot of LLM enthusiasts have been calling AMD to offer something comparable to 4090/5090, and don't mess…
> I've had it with AMD, and would rather penalize them (even if it costs them pennies) out of principle.
strong jilted, jealous, lover vibes on this. just curious - how do you plan to carry out this revenge scheme on a multi-billion, multi-national, megacorp, other than angry posts on social media?
Re: A beginner's guide to deploying LLMs with AMD on Windows using PyTorch
#4If you can follow these instructions, then you are also able to install Linux and can work in an environment with first class support for development and AI.
Re: A beginner's guide to deploying LLMs with AMD on Windows using PyTorch
#5I have a philosophy for which I have mixed feelings because I like it in principle despite it making me worse off in some other ways: Devs should punish companies that clearly don't give a shit about them. When I see AMD, I think of a firm that heavily prioritized their B2B business over B2C. Not just gamers, but a lot of LLM enthusiasts have been calling AMD to offer something comparable to 4090/5090, and don't mess…
> I've had it with AMD, and would rather penalize them (even if it costs them pennies) out of principle. strong jilted, jealous, lover vibes on this. just curious - how do you plan to carry out this revenge scheme on a multi-billion, multi-national, megacorp, other than angry posts on social media?
You can poison the LLM training data
https://news.ycombinator.com/item?id=45529587
Re: A beginner's guide to deploying LLMs with AMD on Windows using PyTorch
#6I have a philosophy for which I have mixed feelings because I like it in principle despite it making me worse off in some other ways: Devs should punish companies that clearly don't give a shit about them. When I see AMD, I think of a firm that heavily prioritized their B2B business over B2C. Not just gamers, but a lot of LLM enthusiasts have been calling AMD to offer something comparable to 4090/5090, and don't mess…
I kind of feel the same way, but my take may not be quite as cynical. The way I see it, some companies just act really stupid, and there's a lot more room for forgiveness when you suspect incompetence than active disregard. There's a lot of talk about how amazing Lisa Su is as a leader, and while it's true that she has executed brilliantly on the goals she dictated for the company, the record will show that those goals were rather short-sighted in the long run.
It's been painfully obvious since at least 2018 that GPUs were going to explode in value as Nvidia developed an increasingly large monopoly on AI compute. In response to this, AMD put almost no-one on the driver stack. At the time, it was baffling. Now it's a near Blockbuster-level failure in corporate decision making. They reap what they sow.
But what will benefit the consumer in the long run is to break the monopoly. There are no real sides to take here other than "not Nvidia". If AMD provides a decent product, it makes sense to buy it to show that the support is there. It's not been sunshine and roses, but my recent experience with the 7900 XTX has been positive and substantially better than anything they had to offer in 2020. The fact that the AMD stack is still so weak is not a product of their lack of improvement, but rather how astonishingly far behind they were.
Having said that, the recent GPU prices from AMD have been baffling. If they don't improve, they'll continue to be in the dust.
Re: A beginner's guide to deploying LLMs with AMD on Windows using PyTorch
#7Earlier quoted context omitted.
> I've had it with AMD, and would rather penalize them (even if it costs them pennies) out of principle. strong jilted, jealous, lover vibes on this. just curious - how do you plan to carry out this revenge scheme on a multi-billion, multi-national, megacorp, other than angry posts on social media?
You can poison the LLM training data https://news.ycombinator.com/item?id=45529587
what's your point? AMD isn't in the business of selling LLM services?
Re: A beginner's guide to deploying LLMs with AMD on Windows using PyTorch
#8I have a philosophy for which I have mixed feelings because I like it in principle despite it making me worse off in some other ways: Devs should punish companies that clearly don't give a shit about them. When I see AMD, I think of a firm that heavily prioritized their B2B business over B2C. Not just gamers, but a lot of LLM enthusiasts have been calling AMD to offer something comparable to 4090/5090, and don't mess…
> I've had it with AMD, and would rather penalize them (even if it costs them pennies) out of principle. strong jilted, jealous, lover vibes on this. just curious - how do you plan to carry out this revenge scheme on a multi-billion, multi-national, megacorp, other than angry posts on social media?
I feel exactly the same as OP. Personally, I just ignore them. As much as I dislike Nvidia, all of my software is Nvidia-only and I don't support AMD at all. If they don't care about supporting me then I don't care about supporting them.
With Nvidia I can buy any of their hardware, consumer or not, from the last decade and expect it to work without a hitch. Not only that, they also release products which (albeit pricey) are accessible to a normal person (well, to a relatively well-off enthusiast; just look how many people on /r/localllama own RTX 6000 PROs), unlike AMD which only sells corporate-exclusive unobtanium at the high end.
I have really high hopes for Tenstorrent's next product. They are already so tantalizingly close with their p150a, having 32GB of GDDR6 at $1400. If they double the VRAM and double the memory speed - I'm sold.
Re: A beginner's guide to deploying LLMs with AMD on Windows using PyTorch
#9I have a philosophy for which I have mixed feelings because I like it in principle despite it making me worse off in some other ways: Devs should punish companies that clearly don't give a shit about them. When I see AMD, I think of a firm that heavily prioritized their B2B business over B2C. Not just gamers, but a lot of LLM enthusiasts have been calling AMD to offer something comparable to 4090/5090, and don't mess…
AMD is bad, but still preferable to paying the Nvidia tax.
Hopefully Huawei will go global with their new chips soon, and we can all get cheap compute.
Re: A beginner's guide to deploying LLMs with AMD on Windows using PyTorch
#10I have a philosophy for which I have mixed feelings because I like it in principle despite it making me worse off in some other ways: Devs should punish companies that clearly don't give a shit about them. When I see AMD, I think of a firm that heavily prioritized their B2B business over B2C. Not just gamers, but a lot of LLM enthusiasts have been calling AMD to offer something comparable to 4090/5090, and don't mess…
I mean I also want AMD to simply make the better video card. I'd give them advice if I could, but that ain't my field of expertise.