71–80 of 362 posts
Run an incredible 400B parameters on a handheld device. 0.6 t/s, wait 30 seconds to see what these billions of calculations get us: "That is a profound observation, and you are absolutely right ..."
I don't think we are ever going to win this. The general population loves being glazed way too much.
"You are absolutely right to be confused"
That was the closest AI has been to calling me "dumb meatbag".
I have some macro opinions about Apple - not sure if I'm correct, but tell me what you think. Apple has always seen RAM as an economic advantage for their platform: Make the development effort to ensure that the OS and apps work well with minimal memory and save billions every year in hardware costs. In 2026, iPhones still come with 8Gb of RAM, Pro/Max come with 12Gb. The problem is that AI (ML/LLM training and infer…
Earlier quoted context omitted.
I think for many reasons this will become the dominant paradigm for end user devices. Moore's law will shrink it to 8mm soon. I think it'll be like a microSD card you plug in. Or we develop a new silicon process that can mimic synaptic weights in biology. Synapses have plasticity.
One big bottleneck is SRAM cost. Even an 8b model would probably end up being hundreds of dollars to run locally on that kind of hardware. Especially unpalatable if the model quality keeps advancing year-by-year. > Or we develop a new silicon process that can mimic synaptic weights in biology. Synapses have plasticity. It's amazing to me that people consider this to be more realistic than FAANG collaborating on a CUD…
Not for this approach
I have some macro opinions about Apple - not sure if I'm correct, but tell me what you think. Apple has always seen RAM as an economic advantage for their platform: Make the development effort to ensure that the OS and apps work well with minimal memory and save billions every year in hardware costs. In 2026, iPhones still come with 8Gb of RAM, Pro/Max come with 12Gb. The problem is that AI (ML/LLM training and infer…
I have some macro opinions about Apple - not sure if I'm correct, but tell me what you think. Apple has always seen RAM as an economic advantage for their platform: Make the development effort to ensure that the OS and apps work well with minimal memory and save billions every year in hardware costs. In 2026, iPhones still come with 8Gb of RAM, Pro/Max come with 12Gb. The problem is that AI (ML/LLM training and infer…
Possibly this just isn't the generation of hardware to solve this problem in? We're like, what three or four years in at most, and only barely two in towards AI assisted development being practical. I wouldn't want to be the first mover here, and I don't know if it's a good point in history to try and solve the problem. Everything we're doing right now with AI, we will likely not be doing in five years. If I were run…
It's not like Apple's GPU designs are world-class anyways, they're basically neck-and-neck with AMD for raster efficiency. Except unlike AMD, Apple has all the resources in the world to compete with Nvidia and simply chooses to sit on their ass.
Earlier quoted context omitted.
A similar approach was recently featured here: https://news.ycombinator.com/item?id=47476422 Though iPhone Pro has very limited RAM (12GB total) which you still need for the active part of the model. (Unless you want to use Intel Optane wearout-resistant storage, but that was power hungry and thus unsuitable to a mobile device.)
> Though iPhone Pro has very limited RAM (12GB total) which you still need for the active part of the model. This is why mixture of experts (MoE) models are favored for these demos: Only a portion of the weights are active for each token.
Earlier quoted context omitted.
One big bottleneck is SRAM cost. Even an 8b model would probably end up being hundreds of dollars to run locally on that kind of hardware. Especially unpalatable if the model quality keeps advancing year-by-year. > Or we develop a new silicon process that can mimic synaptic weights in biology. Synapses have plasticity. It's amazing to me that people consider this to be more realistic than FAANG collaborating on a CUD…
> bottleneck is SRAM cost Not for this approach
Earlier quoted context omitted.
Apple is already one of the winners of the AI race. It’s making much more profit (ie it ain’t losing money) on AI off of ChatGPT, Claude, Grok (you would be surprised at how many incels pay to make AI generated porn videos) subscriptions through the App Store. It’s only paying Google $1 billion a year for access to Gemini for Siri
Plus all those pricey 512GB Mac Studios they are selling to YouTubers.
The financial math on actually buying over $40k worth of Mac for 1 to 2 youtube videos probably doesn't work that well, even for the really big players.