421–430 of 510 posts
Earlier quoted context omitted.
OpenAI doesn’t make any profit. So either it dies or prices go up. Not to mention the privacy aspect of your own machine and the freedom of choice which models to run
> So either it dies or prices go up. Or efficiency gains in hardware and software catchup making current price point profitable.
I'm looking at my Jetson Nano in the corner which is fulfilling its post-retirement role as a paper weight because Nvidia abandoned it in 4 years. Nvidia Jetson Nano, A SBC for "AI" debuted with already aging custom Ubuntu 18.04 and when 18.04 went EOL, Nvidia abandoned it completely without any further updates to its proprietary jet-pack or drivers and without them all of Machine Learning stack like CUDA, Pytorch et…
Is there any recent, powerful SBC with fully upstream kernel support? I can only think of raspberry pi...
>> The IBM Roadrunner was the first supercomputer to reach one petaflop (1 quadrillion floating point operations per second, or FLOPS) on May 25, 2008. $100M, 2.35MW, 6000 ft^2 >>Designed for AI researchers, data scientists, and students, Project Digits packs Nvidia’s new GB10 Grace Blackwell Superchip, which delivers up to a petaflop of computing performance for prototyping, fine-tuning, and running AI models. $3000…
Digits is petaflops of FP4, roadrunner is petaflops of FP32. So at least a factor of 8 difference, but in practice much more. (IE I strongly doubt digits can do 1/8th petaflop of FP32) Beyond that, the factors seem reasonable for 2 decades?
Isn't it actually FP64?
Earlier quoted context omitted.
The bigger picture is that OpenAI o3/o4.. plus specialized models will blow open the doors to genome tagging and discovery, but that is still 1 to 3 years away for ASI to kick in.
While I kinda agree with you, I don't think we will ever find a meaningful way to throw genome sequencing data at LLMs. It's simple too much data. I've worked in a project some years ago where we were using data from genome sequencing of a bacteria. Every sequenced sample was around 3GB of data and sample size was pretty small with only about 100 samples to study. I think the real revolution will happen because code…
I feel this is bigger than the 5x series GPUs. Given the craze around AI/LLMs, this can also potentially eat into Apple’s slice of the enthusiast AI dev segment once the M4 Max/Ultra Mac minis are released. I sure wished I held some Nvidia stocks, they seem to be doing everything right in the last few years!
Apple M chips are pretty efficient.
Welcome to tomorrow's "personal" computer, a single unmodifiable SoC with closed source software stack.
Earlier quoted context omitted.
If you're expecting this device to stay relevant for 4 years you are not the target demographic. Compute is evolving way too rapidly to be setting-and-forgetting anything at the moment.
Eh? By all indications compute is now evolving SLOWER than ever. Moore's Law is dead, Dennard scaling is over, the latest fab nodes are evolutionary rather than revolutionary. This isn't the 80s when compute doubled every 9 months, mostly on clock scaling.
I believe Nvidia has some published numbers for the 5000 series that showed DLSS off performance, which allowed a fair comparison to the previous generation, on the order of 25%, then removed it.
Thankfully the 3rd party benchmarks that use the same settings on old and new hardware should be out soon.
Earlier quoted context omitted.
I'm pretty frugal, but my first thought is to get two to run 405B models. Building out 128GB of VRAM isn't easy, and will likely cost twice this.
You can get a M4 Max MBP with 128GB for $1k less than two of these single-use devices.
Earlier quoted context omitted.
The developers they are referring to aren’t just enthusiasts; they are also developers who were purchasing SuperMicro and Lambda PCs to develop models for their employers. Many enterprises will buy these for local development because it frees up the highly expensive enterprise-level chip for commercial use. This is a genius move. I am more baffled by the insane form factor that can pack this much power inside a Mac M…
How does it run 400B models across two? I didn’t see that in the article
Earlier quoted context omitted.
It's annoying I do LLMs for work and have a bit of an interest in them and doing stuff with GANS etc. I have a bit of an interest in games too. If I could get one platform for both, I could justify 2k maybe a bit more. I can't justify that for just one half: running games on Mac, right now via Linux: no thanks. And on the PC side, nvidia consumer cards only go to 24gb which is a bit limiting for LLMs, while being ver…
The new $2k card from Nvidia will be 32GB but your point stands. AMD is planning a unified chiplet based GPU architecture (AI/data center/workstation/gaming) called UDNA, which might alleviate some of these issues. It's been delayed and delayed though - hence the lackluster GPU offerings from team Red this cycle - so I haven't been getting my hopes up. Maybe (LP)CAMM2 memory will make model usage just cheap enough th…
I do hope that a AMD Strix Halo ships with 2 LPCAMM2 slots for a total width of 256 bits.