451–460 of 510 posts
Earlier quoted context omitted.
How close does any of that get a person to having Ubuntu 24.04 running on their board? (I guess we can put aside the issue of Nvidia's closed source graphics drivers for the moment)
Isn't the Jetson line more of an embedded line and not a end-user desktop? Why would you run Ubuntu?
In any case, Ubuntu is what it comes with.
Earlier quoted context omitted.
my $5k m3 max 128gb disagrees
Doubt it, a year ago useful local LLMs on a Mac (via something like ollama) was barely taking off. If what you say it's true you were among the first 100 people on the planet who were doing this; which btw, further supports my argument on how extremely rare is that use case for Mac users.
What are the CPU specs? Idk about the GPU but a really fast ARM cpu and a ton of ram and it already runs Linux?!! If it’s competitive with the M chips from Apple this might be my next box.
10 cortex-x925 and 10 cortex-a725 I believe.
I feel this is bigger than the 5x series GPUs. Given the craze around AI/LLMs, this can also potentially eat into Apple’s slice of the enthusiast AI dev segment once the M4 Max/Ultra Mac minis are released. I sure wished I held some Nvidia stocks, they seem to be doing everything right in the last few years!
Jensen did say in recent interview, paraphrasing, “they are trying to kill my company”. Those Macs with unified memory is a threat he is immediately addressing. Jensen is a wartime ceo from the looks of it, he’s not joking. No wonder AMD is staying out of the high end space, since NVIDIA is going head on with Apple (and AMD is not in the business of competing with Apple).
I think this is a race that Apple doesn't know it's part of. Apple has something that happens to work well for AI, as a side effect of having a nice GPU with lots of fast shared memory. It's not marketed for inference.
Earlier quoted context omitted.
I mean I can get a Snapdragon X Elite laptop for $1200 that'll have a faster CPU than the one in the Digits, too...
Sure, but less than half the memory bandwidth and inference is largely bandwidth bound.
> lots of people will buy these machines to get an AArch64 Linux workstation—even if they are not interested in AI or Nvidia GPUs.
Earlier quoted context omitted.
Jensen did say in recent interview, paraphrasing, “they are trying to kill my company”. Those Macs with unified memory is a threat he is immediately addressing. Jensen is a wartime ceo from the looks of it, he’s not joking. No wonder AMD is staying out of the high end space, since NVIDIA is going head on with Apple (and AMD is not in the business of competing with Apple).
Which interview was this?
I can't find the exact Youtube video, but it's out there.
Earlier quoted context omitted.
Sure, but less than half the memory bandwidth and inference is largely bandwidth bound.
The context was people that don't care about AI or Nvidia GPUs and just want an AArch64 system. So inference performance is irrelevant here > lots of people will buy these machines to get an AArch64 Linux workstation—even if they are not interested in AI or Nvidia GPUs.
Earlier quoted context omitted.
Isn’t the cloud GPU market covering this? I can run a model for $2/hr, or get a 8xH100 if I need to play with something bigger.
People tend to limit their usage when it's time-billed. You need some sort of desktop computer anyway, so, if you spend the 3K this one costs, you have unlimited time of Nvidia cloud software. When you need to run on bigger metal, then you pay $2/hour.
Will there be a healthy "personal AI supercomputer" economy to generate demand for this? (NB: spam generators are only a parasite on any digital economy, viable to the extend they don't kill the host). One can only wish for this, but Nvidia would be going against the decades-long trend to emaciate local computing in favor of concentrating all compute on somebody else's linux (aka: cloud).
Also I consider this a dev board. Soon this tech will be everywhere, in our phones, computers...
You could already plug that to your home assistant and have your own Star Trek computer you can ask questions from. And NVIDIA seems to know this is the future and they were the first in the market.
Earlier quoted context omitted.
From https://www.tomshardware.com/pc-components/cpus/amds-beastly... The fire-breathing 120W Zen 5-powered flagship Ryzen AI Max+ 395 comes packing 16 CPU cores and 32 threads paired with 40 RDNA 3.5 (Radeon 8060S) integrated graphics cores (CUs), but perhaps more importantly, it supports up to 128GB of memory that is shared among the CPU, GPU, and XDNA 2 NPU AI engines. The memory can also be carved up to a distinct…
"Fire breathing" is completely inappropriate. Strix Halo is a replacement for the high-power laptop CPUs from the HX series of Intel and AMD, together with a discrete GPU. The thermal design power of a laptop CPU-dGPU combo is normally much higher than 120 W, which is the maximum TDP recommended for Strix Halo. The faster laptop dGPUs want more than 120 W only for themselves, not counting the CPU. So any claims of be…
Normally? Much higher than 120W? Those are some pretty abnormal (and dare I say niche?) laptops you're talking about there. Remember, that's not peak power - thermal design power is what the laptop should be able to power and cool pretty much continuously.
At those power levels, they're usually called DTR: desktop replacement. You certainly can't call it "just a laptop" anymore once we're in needs-two-power-supplies territory.