Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A beginner's guide to deploying LLMs with AMD on Windows using PyTorch

gpuopen.com

Post reply on HN
Print view View original

21–30 of 48 posts

Re: A beginner's guide to deploying LLMs with AMD on Windows using PyTorch

#21

Post by pjmlp »

tuna74 wrote: post #20
pjmlp wrote: post #19

Earlier quoted context omitted.

GNU/Linux like proper laptops and desktops, not gaming handhelds. 100% of the gamers that don't overlap with FOSS folks, don't care SteamOS is somehow based on the Linux kernel, the Steam Runtime and a Win32/DirectX translation layer, otherwise there would not be games to play.

Calling a Linux distro Gnu/Linux today is wrong because the actual Gnu code running is such a small part of everything. SteamOS uses Linux as much as Linux distros like Fedora and Arch and other Linux based systems like Android. A lot of gamers are also FOSS folks although not as hardcore as RMS.

Not at all, it makes a point, ChromeOS, WebOS, Android, are also based on the Linux kernel and hardly considered Linux distributions for everyday use, in a way similar to GNU/Linux expectations.

How little do game studios care to port their titles from Android/NDK into SteamOS, even though both are "Linux".

Re: A beginner's guide to deploying LLMs with AMD on Windows using PyTorch

#22

Post by yonaguska »

behnamoh wrote: post #2

I have a philosophy for which I have mixed feelings because I like it in principle despite it making me worse off in some other ways: Devs should punish companies that clearly don't give a shit about them. When I see AMD, I think of a firm that heavily prioritized their B2B business over B2C. Not just gamers, but a lot of LLM enthusiasts have been calling AMD to offer something comparable to 4090/5090, and don't mess…

I haven't been following the market until recently, but isn't the AMD Ryzen Max AI a consumer friendly AI option? Is it just not serious enough relative to the Nvidia offerings?

Re: A beginner's guide to deploying LLMs with AMD on Windows using PyTorch

#23

Post by pjmlp »

drnick1 wrote: post #4

If you can follow these instructions, then you are also able to install Linux and can work in an environment with first class support for development and AI.

These instructions assume there is already a working OS where everything is supported.

Some of us consider modern IDEs and advanced debugging tools a much better support for development.

https://bitsavers.org/pdf/xerox/parc/techReports/CSL-89-4_UN...

Re: A beginner's guide to deploying LLMs with AMD on Windows using PyTorch

#24

Post by fransje26 »

buyucu wrote: post #9
behnamoh wrote: post #2

I have a philosophy for which I have mixed feelings because I like it in principle despite it making me worse off in some other ways: Devs should punish companies that clearly don't give a shit about them. When I see AMD, I think of a firm that heavily prioritized their B2B business over B2C. Not just gamers, but a lot of LLM enthusiasts have been calling AMD to offer something comparable to 4090/5090, and don't mess…

AMD is bad, but still preferable to paying the Nvidia tax. Hopefully Huawei will go global with their new chips soon, and we can all get cheap compute.

Cue the 1000% tariff on said affordable Huawei chip.

Re: A beginner's guide to deploying LLMs with AMD on Windows using PyTorch

#25

Post by LogicFailsMe »

janalsncm wrote: post #12
hodgehog11 wrote: post #6

Earlier quoted context omitted.

I kind of feel the same way, but my take may not be quite as cynical. The way I see it, some companies just act really stupid, and there's a lot more room for forgiveness when you suspect incompetence than active disregard. There's a lot of talk about how amazing Lisa Su is as a leader, and while it's true that she has executed brilliantly on the goals she dictated for the company, the record will show that those goa…

What will likely break the monopoly is a Chinese competitor producing a worse but usable product within the next couple of years. AMD will be the next Compaq.

Indeed, but it will take more than a few years, the entire industry negging GPUs for a decade and half comes at a high price for them. And looking back historically, dominant companies like Intel had been sucking wind for over a decade before its ineptitude caught up with it. Microsoft was repeatedly punching itself in the face until Satya Nadella took over and pulled it back from the brink just by getting them to stop doing so. And a quarter century later, CSCO has nearly recovered to its dotcom peak stock value.

But Nvidia isn't inept yet, and despite some of its people being just as arrogant as Googlers at Google's prime, they still deliver a vastly better product and ecosystem than anyone else. The Chinese will crack that, but the first few generations are going to be a rough ride. Just check out the iterations on any knockoff of DeWalt tools they sell with cheaper parts until they nail the price/performance ratio to perfection to see how they roll. But also, Nvidia remains a moving target with near bottomless pockets now. Good luck with that.

Re: A beginner's guide to deploying LLMs with AMD on Windows using PyTorch

#26

Post by tokai »

pjmlp wrote: post #17
behnamoh wrote: post #2

I have a philosophy for which I have mixed feelings because I like it in principle despite it making me worse off in some other ways: Devs should punish companies that clearly don't give a shit about them. When I see AMD, I think of a firm that heavily prioritized their B2B business over B2C. Not just gamers, but a lot of LLM enthusiasts have been calling AMD to offer something comparable to 4090/5090, and don't mess…

They love gamers, just not on GNu/Linux. Playstation, XBox, Windows, gaming handhelds.

What do you mean, AMD has been the go too for linux gamers for a long time.

Re: A beginner's guide to deploying LLMs with AMD on Windows using PyTorch

#27

Post by curt15 »

yonaguska wrote: post #22
behnamoh wrote: post #2

I have a philosophy for which I have mixed feelings because I like it in principle despite it making me worse off in some other ways: Devs should punish companies that clearly don't give a shit about them. When I see AMD, I think of a firm that heavily prioritized their B2B business over B2C. Not just gamers, but a lot of LLM enthusiasts have been calling AMD to offer something comparable to 4090/5090, and don't mess…

I haven't been following the market until recently, but isn't the AMD Ryzen Max AI a consumer friendly AI option? Is it just not serious enough relative to the Nvidia offerings?

Does ROCm work on Ryzem Max AI?

Re: A beginner's guide to deploying LLMs with AMD on Windows using PyTorch

#28

Post by jakogut »

curt15 wrote: post #27
yonaguska wrote: post #22

Earlier quoted context omitted.

I haven't been following the market until recently, but isn't the AMD Ryzen Max AI a consumer friendly AI option? Is it just not serious enough relative to the Nvidia offerings?

Does ROCm work on Ryzem Max AI?

It works on the GPU, and the NPU is supposed to work on Windows using a framework called lemonade (I haven't tried), but the NPU is not supported with the same software stack on Linux yet.

Re: A beginner's guide to deploying LLMs with AMD on Windows using PyTorch

#29

Post by pjmlp »

tokai wrote: post #26
pjmlp wrote: post #17

Earlier quoted context omitted.

They love gamers, just not on GNu/Linux. Playstation, XBox, Windows, gaming handhelds.

What do you mean, AMD has been the go too for linux gamers for a long time.

OP was talking about which customers AMD actually cares about, apparently not enough about GNU/Linux gamers.

Re: A beginner's guide to deploying LLMs with AMD on Windows using PyTorch

#30

Post by _aavaa_ »

pjmlp wrote: post #19
_aavaa_ wrote: post #18

Earlier quoted context omitted.

The steam deck is famously Linux and uses AMD

GNU/Linux like proper laptops and desktops, not gaming handhelds. 100% of the gamers that don't overlap with FOSS folks, don't care SteamOS is somehow based on the Linux kernel, the Steam Runtime and a Win32/DirectX translation layer, otherwise there would not be games to play.

What, prey tell, is a “proper” laptop and desktop?

Plug a usb dock into the steam deck and you have a “proper” computer.

Post reply on HN