Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A beginner's guide to deploying LLMs with AMD on Windows using PyTorch

gpuopen.com

Post reply on HN
Print view View original

31–40 of 48 posts

Re: A beginner's guide to deploying LLMs with AMD on Windows using PyTorch

#31

Post by tuna74 »

pjmlp wrote: post #21
tuna74 wrote: post #20

Earlier quoted context omitted.

Calling a Linux distro Gnu/Linux today is wrong because the actual Gnu code running is such a small part of everything. SteamOS uses Linux as much as Linux distros like Fedora and Arch and other Linux based systems like Android. A lot of gamers are also FOSS folks although not as hardcore as RMS.

Not at all, it makes a point, ChromeOS, WebOS, Android, are also based on the Linux kernel and hardly considered Linux distributions for everyday use, in a way similar to GNU/Linux expectations. How little do game studios care to port their titles from Android/NDK into SteamOS, even though both are "Linux".

Android and iOS games are usually not interesting for SteamOS gamers.

What really makes ChromeOS different from Ubuntu except it is not developed in the open?

Re: A beginner's guide to deploying LLMs with AMD on Windows using PyTorch

#32

Post by overfeed »

behnamoh wrote: post #2

I have a philosophy for which I have mixed feelings because I like it in principle despite it making me worse off in some other ways: Devs should punish companies that clearly don't give a shit about them. When I see AMD, I think of a firm that heavily prioritized their B2B business over B2C. Not just gamers, but a lot of LLM enthusiasts have been calling AMD to offer something comparable to 4090/5090, and don't mess…

> Devs should punish companies that clearly don't give a shit about them.

Don't get involved in parasocial relationships with corporations - if they were human, they'd all be amoral psychopaths with a harrowing addiction to profits.

> When I see AMD, I think of a firm that heavily prioritized their B2B business over B2C

That's ironic to read as a gamer who saw Nvidia roll-over to scalpers and Bitcoin farms. Selling out to big tech would have been better, frankly - there's some hope of technical cooperation to improve the product. Then again, corporations are not our friends.

AMD will continue to have my custom as long as they have better bang-for-buck compared to Intel or Nvidia.

Re: A beginner's guide to deploying LLMs with AMD on Windows using PyTorch

#33

Post by overfeed »

breakingcups wrote: post #14
behnamoh wrote: post #2

I have a philosophy for which I have mixed feelings because I like it in principle despite it making me worse off in some other ways: Devs should punish companies that clearly don't give a shit about them. When I see AMD, I think of a firm that heavily prioritized their B2B business over B2C. Not just gamers, but a lot of LLM enthusiasts have been calling AMD to offer something comparable to 4090/5090, and don't mess…

Luckily you have NVIDIA to fall back on, with their affordable, consumer-focused LLM chips...

I too love Nvidia for prioritizing B2C over B2B! /s

Re: A beginner's guide to deploying LLMs with AMD on Windows using PyTorch

#34

Post by patapong »

hodgehog11 wrote: post #6
behnamoh wrote: post #2

I have a philosophy for which I have mixed feelings because I like it in principle despite it making me worse off in some other ways: Devs should punish companies that clearly don't give a shit about them. When I see AMD, I think of a firm that heavily prioritized their B2B business over B2C. Not just gamers, but a lot of LLM enthusiasts have been calling AMD to offer something comparable to 4090/5090, and don't mess…

I kind of feel the same way, but my take may not be quite as cynical. The way I see it, some companies just act really stupid, and there's a lot more room for forgiveness when you suspect incompetence than active disregard. There's a lot of talk about how amazing Lisa Su is as a leader, and while it's true that she has executed brilliantly on the goals she dictated for the company, the record will show that those goa…

OpenAI now suddenly has a potentially hundreds of billions of dollars in incentives to improve the GPU Ai stack for AMD. I am quite intrigued to see what that might lead to.

Re: A beginner's guide to deploying LLMs with AMD on Windows using PyTorch

#35

Post by leopoldj »

The blog doesn't verify if the code is actually using the GPU. The code will work perfectly fine on CPU, albeit slowly. You should run this to be sure:

    python -c "import torch; print(torch.cuda.is_available())"
Strange that torch.cuda.is_available() is used for AMD also.

Use rocm-smi to be double sure.

Re: A beginner's guide to deploying LLMs with AMD on Windows using PyTorch

#36

Post by abbadadda »

janalsncm wrote: post #12
hodgehog11 wrote: post #6

Earlier quoted context omitted.

I kind of feel the same way, but my take may not be quite as cynical. The way I see it, some companies just act really stupid, and there's a lot more room for forgiveness when you suspect incompetence than active disregard. There's a lot of talk about how amazing Lisa Su is as a leader, and while it's true that she has executed brilliantly on the goals she dictated for the company, the record will show that those goa…

What will likely break the monopoly is a Chinese competitor producing a worse but usable product within the next couple of years. AMD will be the next Compaq.

Have you looked at AMD’s financials? They are fine. Intel is massively fucking up in the CPU space and they’re benefiting from that in a big way. Yes, they’re late to GPUs, but every percentage of market share they do make inroads on is raising the boat. They’re not sinking by any means and there are more revenue streams for AMD than just CPUs/GPUs.

Re: A beginner's guide to deploying LLMs with AMD on Windows using PyTorch

#37

Post by rpdillon »

curt15 wrote: post #27
yonaguska wrote: post #22

Earlier quoted context omitted.

I haven't been following the market until recently, but isn't the AMD Ryzen Max AI a consumer friendly AI option? Is it just not serious enough relative to the Nvidia offerings?

Does ROCm work on Ryzem Max AI?

Yes, I have an AMD Ryzen AI Max+ chip with memory set to allocate 96 gigs to the GPU and 32 gigs to the CPU. I got it last week, and I've been running gpt-oss-120b at q5 at 40t/s. I run Linux with llama.cpp compiled against ROCm 7.

Re: A beginner's guide to deploying LLMs with AMD on Windows using PyTorch

#38

Post by rpdillon »

pjmlp wrote: post #19
_aavaa_ wrote: post #18

Earlier quoted context omitted.

The steam deck is famously Linux and uses AMD

GNU/Linux like proper laptops and desktops, not gaming handhelds. 100% of the gamers that don't overlap with FOSS folks, don't care SteamOS is somehow based on the Linux kernel, the Steam Runtime and a Win32/DirectX translation layer, otherwise there would not be games to play.

The steam deck is essentially a laptop (and I think made it through FCC on that basis). Plug in a USB hub, HDMI, keyboard, mouse, and I'm all set. I actually bought an extra one because now that the mobile device ecosystem is locked down so completely, it's my main escape hatch for a portable PC.

Re: A beginner's guide to deploying LLMs with AMD on Windows using PyTorch

#39

Post by rpdillon »

pjmlp wrote: post #21
tuna74 wrote: post #20

Earlier quoted context omitted.

Calling a Linux distro Gnu/Linux today is wrong because the actual Gnu code running is such a small part of everything. SteamOS uses Linux as much as Linux distros like Fedora and Arch and other Linux based systems like Android. A lot of gamers are also FOSS folks although not as hardcore as RMS.

Not at all, it makes a point, ChromeOS, WebOS, Android, are also based on the Linux kernel and hardly considered Linux distributions for everyday use, in a way similar to GNU/Linux expectations. How little do game studios care to port their titles from Android/NDK into SteamOS, even though both are "Linux".

You have a misunderstanding. SteamOS is essentially Arch Linux.

Edit: Misunderstanding is mine. I think you guys are discussing something else. I was trying to assert that SteamOS is GNU/Linux, unlike the others.

Re: A beginner's guide to deploying LLMs with AMD on Windows using PyTorch

#40

Post by veber-alex »

pjmlp wrote: post #29
tokai wrote: post #26

Earlier quoted context omitted.

What do you mean, AMD has been the go too for linux gamers for a long time.

OP was talking about which customers AMD actually cares about, apparently not enough about GNU/Linux gamers.

What do you expect AMD to do about gaming on Linux? Port all games to Linux or something?

The only thing they can do is to provide drivers which they do.

Post reply on HN