Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A beginner's guide to deploying LLMs with AMD on Windows using PyTorch

gpuopen.com

Post reply on HN
Print view View original

41–48 of 48 posts

Re: A beginner's guide to deploying LLMs with AMD on Windows using PyTorch

#41

Post by pjmlp »

tuna74 wrote: post #31
pjmlp wrote: post #21

Earlier quoted context omitted.

Not at all, it makes a point, ChromeOS, WebOS, Android, are also based on the Linux kernel and hardly considered Linux distributions for everyday use, in a way similar to GNU/Linux expectations. How little do game studios care to port their titles from Android/NDK into SteamOS, even though both are "Linux".

Android and iOS games are usually not interesting for SteamOS gamers. What really makes ChromeOS different from Ubuntu except it is not developed in the open?

Userspace is a browser.

Re: A beginner's guide to deploying LLMs with AMD on Windows using PyTorch

#42

Post by pjmlp »

veber-alex wrote: post #40
pjmlp wrote: post #29

Earlier quoted context omitted.

OP was talking about which customers AMD actually cares about, apparently not enough about GNU/Linux gamers.

What do you expect AMD to do about gaming on Linux? Port all games to Linux or something? The only thing they can do is to provide drivers which they do.

Apparently not, which was the point being made by OP.

Re: A beginner's guide to deploying LLMs with AMD on Windows using PyTorch

#43

Post by fooker »

curt15 wrote: post #27
yonaguska wrote: post #22

Earlier quoted context omitted.

I haven't been following the market until recently, but isn't the AMD Ryzen Max AI a consumer friendly AI option? Is it just not serious enough relative to the Nvidia offerings?

Does ROCm work on Ryzem Max AI?

Barely.

Re: A beginner's guide to deploying LLMs with AMD on Windows using PyTorch

#44

Post by fooker »

breakingcups wrote: post #14
behnamoh wrote: post #2

I have a philosophy for which I have mixed feelings because I like it in principle despite it making me worse off in some other ways: Devs should punish companies that clearly don't give a shit about them. When I see AMD, I think of a firm that heavily prioritized their B2B business over B2C. Not just gamers, but a lot of LLM enthusiasts have been calling AMD to offer something comparable to 4090/5090, and don't mess…

Luckily you have NVIDIA to fall back on, with their affordable, consumer-focused LLM chips...

Given that you can buy a 48GB RTX6000 on newegg right now, yes?

Re: A beginner's guide to deploying LLMs with AMD on Windows using PyTorch

#45

Post by PhilippGille »

pjmlp wrote: post #42
veber-alex wrote: post #40

Earlier quoted context omitted.

What do you expect AMD to do about gaming on Linux? Port all games to Linux or something? The only thing they can do is to provide drivers which they do.

Apparently not, which was the point being made by OP.

OP talked about LLMs, not gaming. It's a different software stack.

Re: A beginner's guide to deploying LLMs with AMD on Windows using PyTorch

#46

Post by a96 »

z3ratul163071 wrote: post #11
behnamoh wrote: post #2

I have a philosophy for which I have mixed feelings because I like it in principle despite it making me worse off in some other ways: Devs should punish companies that clearly don't give a shit about them. When I see AMD, I think of a firm that heavily prioritized their B2B business over B2C. Not just gamers, but a lot of LLM enthusiasts have been calling AMD to offer something comparable to 4090/5090, and don't mess…

it's not only the support. they are unable to write good software. period.

They're a hardware company. They should stay the hell away from software and just publish specs so actual software people can do it properly.

Re: A beginner's guide to deploying LLMs with AMD on Windows using PyTorch

#47

Post by a96 »

behnamoh wrote: post #2

I have a philosophy for which I have mixed feelings because I like it in principle despite it making me worse off in some other ways: Devs should punish companies that clearly don't give a shit about them. When I see AMD, I think of a firm that heavily prioritized their B2B business over B2C. Not just gamers, but a lot of LLM enthusiasts have been calling AMD to offer something comparable to 4090/5090, and don't mess…

And the problem with this attitude is that it keeps driving people toward a greater evil like nvidia, intel or the chinese communist party.

AMD is the least evil option so far. We're all disappointed in them, but at least they still exist.

Re: A beginner's guide to deploying LLMs with AMD on Windows using PyTorch

#48

Post by lostmsu »

rpdillon wrote: post #37
curt15 wrote: post #27

Earlier quoted context omitted.

Does ROCm work on Ryzem Max AI?

Yes, I have an AMD Ryzen AI Max+ chip with memory set to allocate 96 gigs to the GPU and 32 gigs to the CPU. I got it last week, and I've been running gpt-oss-120b at q5 at 40t/s. I run Linux with llama.cpp compiled against ROCm 7.

Did you try the native mxfp4 (obviously, Vulkan/ROCm would have to load and upscale it)?
Post reply on HN