431–440 of 510 posts
Earlier quoted context omitted.
I would bet that people running LLMs on their Macs, today, is <0.1% of their user base.
People buying Macs for LLMs—sure I agree. Since the current MacOS comes built in with small LLMs, that number might be closer to 50% not 0.1%.
I feel this is bigger than the 5x series GPUs. Given the craze around AI/LLMs, this can also potentially eat into Apple’s slice of the enthusiast AI dev segment once the M4 Max/Ultra Mac minis are released. I sure wished I held some Nvidia stocks, they seem to be doing everything right in the last few years!
> they seem to be doing everything right in the last few years About that... Not like there isn't a lot to be desired from the linux drivers: I'm running a K80 and M40 in a workstation at home and the thought of having to ever touch the drivers, now that the system is operational, terrifies me. It is by far the biggest "don't fix it if it ain't broke" thing in my life.
Earlier quoted context omitted.
Now that the majority of their revenue is from data centers instead of Windows gaming PCs, you'd think their relationship with Linux should improve or already has.
They have been making real improvements the last few years. Most of their proprietary driver code is in firmware now, and the kernel driver is open-source[1] (the userland-side is still closed though). They've also significantly improved support for wayland and stopped trying to force eglstreams on the community. Wayland+nvidia works quite well now, especially after they added explicit sync support. 1. https://github…
... as in remember the time a ransomware hacker outfit demanded they release the drivers or else .....
https://www.webpronews.com/open-source-drivers-or-else-nvidi...
Earlier quoted context omitted.
If I were NVidia, I would be throwing everything I could at making entertainment experiences that need one of these to run... I mean, this is awfully close to being "Her" in a box, right?
I feel like a lot of people miss that Her was a dystopian future, not an ideal to hit. Also, it’s $3000. For that you could buy subscriptions to OpenAI etc and have the dystopian partner everywhere you go.
Earlier quoted context omitted.
Isn’t the cloud GPU market covering this? I can run a model for $2/hr, or get a 8xH100 if I need to play with something bigger.
I have the skills to write efficient CUDA kernels, but $2/hr is 10% of my salary, so no way I'm renting any H100s. The electricity price for my computer is already painful enough as is. I am sure there are many eastern European developers who are more skilled and get paid even less. This is a huge waste of resources all due to NVIDIA's artificial market segmentation. Or maybe I am just cranky because I want more VRAM…
A real shame it's not running mainline Linux - I don't like their distro based on Ubuntu LTS.
Earlier quoted context omitted.
Are you aware that mainline linux runs on these Jetson devices? It's a bit of annoying work, but you can be running ArchLinuxARM. https://github.com/archlinuxarm/PKGBUILDs/pull/1580 Edit: It's been a while since I did this, but I had to manually build the kernel, overwrite a dtb file maybe (and Linux_for_Tegra/bootloader/l4t_initrd.img) and run something like this (for xavier) sudo ./flash.sh -N 128.30.84.100:/srv/ar…
How close does any of that get a person to having Ubuntu 24.04 running on their board? (I guess we can put aside the issue of Nvidia's closed source graphics drivers for the moment)
$3k for a 128GB standalone is quite favorable pricing considering the next best option at home is going to be a 32GB 5090 at $2k for the card alone, so probably $3k when you’re done building a rig around it.
Earlier quoted context omitted.
This is something every company should make sure they have: an onboarding path. Xeon Phi failed for a number of reasons, but one where it didn't need to fail was availability of software optimised for it. Now we have Xeons and EPYCs, and MI300C's with lots of efficient cores, but we could have been writing software tailored for those for 10 years now. Extracting performance from them would be a solved problem at this…
Raptor Computing provides POWER9 workstations. They're not cheap, still use last-gen hardware (DDR4/PCIe 4 ... and POWER9 itself) but they're out there. https://www.raptorcs.com/content/base/products.html
I feel this is bigger than the 5x series GPUs. Given the craze around AI/LLMs, this can also potentially eat into Apple’s slice of the enthusiast AI dev segment once the M4 Max/Ultra Mac minis are released. I sure wished I held some Nvidia stocks, they seem to be doing everything right in the last few years!
This is something every company should make sure they have: an onboarding path. Xeon Phi failed for a number of reasons, but one where it didn't need to fail was availability of software optimised for it. Now we have Xeons and EPYCs, and MI300C's with lots of efficient cores, but we could have been writing software tailored for those for 10 years now. Extracting performance from them would be a solved problem at this…
Earlier quoted context omitted.
Digits is petaflops of FP4, roadrunner is petaflops of FP32. So at least a factor of 8 difference, but in practice much more. (IE I strongly doubt digits can do 1/8th petaflop of FP32) Beyond that, the factors seem reasonable for 2 decades?
> roadrunner is petaflops of FP32 Isn't it actually FP64?