51–60 of 312 posts
Earlier quoted context omitted.
Yeah, nvidia linux support is meh, but still much better than amd.
Is it better than AMD? I have had literally no graphics issues on my 6650 XT with swaywm using the built in kernel drivers.
Best of all is that I simply set the device to `torch.device('cuda')` rather than openCL, which does wonders for compatibility and to keep code simple.
Protip: Use the official ROCM Pytorch base docker image [0]. The AMD setup is so finicky and dependent on specific versions of sdk/drivers/libraries and it will be much harder to make work if you try to install them separately.
[0]: https://rocm.docs.amd.com/en/latest/how_to/pytorch_install/p...
And the question for most that remains once AMD catches up: will the duopoly result in lower prices to a reasonable level for hobbyists or bootstrapped startups, or will AMD just gouge like NVidia?
Earlier quoted context omitted.
Yeah, nvidia linux support is meh, but still much better than amd.
Is it better than AMD? I have had literally no graphics issues on my 6650 XT with swaywm using the built in kernel drivers.
Yup, thank the hobbyists. Pytorch is allowing other hardware. Stable diffusion working on m chips, intel arc, and Amd. Now what I'd like to see is real benchmarks for compute power. Might even get a few startups to compete in this new area.
Hobbyist and open-source are definitely not synonyms.
CUDA is the only reason I have an Nvidia card, but if more projects start migrating to a more agnostic environment, I'll be really grateful. Running Nvidia in Linux isn't as much fun. Fedora and Debian can be incredibly reliable systems, but when you add an Nvidia card, I feel like I am back in Windows Vista with kernel crashes from time to time.
Yeah, nvidia linux support is meh, but still much better than amd.
I have a nvidia laptop with popos. That works well.
Earlier quoted context omitted.
>They are just too late even if they catch up. Late certainly, too late I don't think so. If you can field a competitively priced consumer card that can run llama fast then you're already halfway there because then the ecosystem takes off. Especially since nvidia is being really stingy with their vram amounts. H100 & datacenter is a separate battle certainly, but on mindshare I think some deft moves from AMD will get…
>If you can field a competitively priced consumer card if this unicorn were to show up, what's to say that all the non-consumers won't just scarf up these equally performant yet lower priced cards causing the supply-demand situation we're in now? the only difference would be a sudden supply of the expensive Nvidia cards that nobody wants because of their price.
A unicorn like that showed up a couple hours ago. Someone posted a guide for getting llama to run on a 7900xtx
https://old.reddit.com/r/LocalLLaMA/comments/170tghx/guide_i...
It's still slow and janky but this really isn't that far away.
I don't buy that AMD can't make this happen if they actually tried.
Go on fiverr, get them to compile a list of top 100 people in the DIY LLM space, send them all free 7900XTXs. Doesn't matter if half of it is wrong, just send it. Next take 1.2m USD, post a dozen 100k bounties against llama.cpp that are AMD specific - support & optimise the gear. Rinse and repeat with every other hobbyist LLM/stable diffusion project. A lot of these are zero profit open source / passion / hobby projects. If 6 figure bounties show up it'll absolute raise pulses. Next do all the big youtubers in the space - carefully on that one so that it doesn't come across as an attempted pay-off...but you want them to know that you want this space to grow and are willing to put your money where your mouth is.
That'll cost AMD what 2m 3m? To move the needle on a multi billion market? That's the cheapest marketing you've ever seen.
As I said the datacenter & enterprise market is another beast entirely full of moats and strategy, but I don't see why a suitably motivated senior AMD exec can't tackle the enthusiast market single handedly with a couple of emails, a cheque book and a tshirt that has the nike slogan on it.
>what's to say that all the non-consumers won't just scarf up these equally performant yet lower priced cards
It doesn't matter. They're in the business of selling cards. To consumers, to datacenters, to your grandmother. From a profit driven capitalist company the details don't matter as long as there is traction & volume. The above - opening up even the possibility of a new market - is gold in that perspective. And from a consumer perspective anything that breaks the nvidia cuda monopoly is a win.
It's worth noting that AMD also has a ROCm port of Tensorflow.
When I try to install rocm-ml-sdk on Arch linux it'll tell me the total installed size would be about 18GB. What can possibly explain this much bloat for what should essentially be a library on top of a graphics driver as well as some tools (compiler, profiler etc.)? A couple hundred MB I could understand if they come with graphical apps and demos, but not this..
Still, if you're right that this package seems to take 18 GB disk size, something weird is going on.
Earlier quoted context omitted.
Didn't he do what he always does. Rake in a ton of money, fart around and then cash out exclaiming it's everyone else's fault? The way he stole Fail0verflow's work with the PS3 security leak after failing to find a hypervisor exploit for months absolutely soured any respect I had for him at the time
Do you have a source on the stealing part? A quick Google search didn't result in anything
Fail0verflow demoed how they were able to derive the private signing keys for the Sony Playstation 3 console at I believe CCC
Geohot after watching the livestream raced into action to demo a "hello world!" jailbreak application and absolutely stole their thunder without giving any credit
Earlier quoted context omitted.
Yeah, nvidia linux support is meh, but still much better than amd.
Is it better than AMD? I have had literally no graphics issues on my 6650 XT with swaywm using the built in kernel drivers.