[flagged]
Apple Foundation Models
131–140 of 244 posts
Re: Apple Foundation Models
#132> a Swift package that makes Claude available as a server-side language model in Apple's Foundation Models framework Ahh I was hoping for the opposite: all of the existing features of Claude Code but somehow running locally on my laptop's neural engine. A pipe dream on an M2 with 8 GB of RAM, but I had a flicker of hope there.
I've found most of the frontier coding models require somewhere between 300GB to 1TB to run with full capabilities.
If only we could buy 1TB of unified memory in a Mac for $1k-$2k in total hardware costs. Apple would basically be able to extinguish the entirety of the market cap for Nvidia, OpenAI, Anthropic, and others all at once.
In 10 years, I hope my MacBook Pro can run today's frontier models and has 1TB of unified Memory.
Re: Apple Foundation Models
#133Earlier quoted context omitted.
> I think Evans is completely wrong. There are only 2 truly frontier models. (at least for now). And Anthropic seems to be leaving OpenAI behind so there might be only 1 in the near future. (which is scary/dangerous) Truly fascinating ecosystem and community in general, as experiences differ so wildly. Anthropic's models seems far behind OpenAI to me, especially when you get into "Pro" territory, and there doesn't se…
People use a model as their daily driver, get very familiar with it and it's behavior, and then go and use another model and have a hard time. It's very difficult to separate "the model is bad" from "the model works differently".
> It's very difficult to separate "the model is bad" from "the model works differently"
At which point it’s fair to reject the commoditization label.
Also missing from these discussions are e.g. Qwen, which is at least as good as one back from OpenAI or Anthropic’s frontiers.
Re: Apple Foundation Models
#134Earlier quoted context omitted.
Benedict Evans may be right after all; frontier models look more and more like telecom companies in the 90s. Billions and billions of investment in infrastructure while others further up the stack captured all the value.
In spite of their deeper pockets, massive datacenters, colosal amounts of user data, and hundreds of thousands of top developers, even Amazon, Meta, Microsoft, and Google are well behind. I think Evans is completely wrong. There are only 2 truly frontier models. (at least for now). And Anthropic seems to be leaving OpenAI behind so there might be only 1 in the near future. (which is scary/dangerous)
Remember the implicit “pareto” in “frontier models”.
Anthropic and OpenAI are far behind state of the art for the entire curve except the “extremely expensive for barely measurable improvements” part.
GLM is probably the third most expensive frontier model (benchmarks and reviews will say for sure), and is apparently ~Opus 4.6 for 10% the inference cost.
The last I checked, qwen was still owning the 24-32GiB RAM range (it runs reasonably without a GPU!) and somewhere around 3.5-4 generation models.
Also, even anthropic says Mythos ~= ChatGPT 5.5, so it’s unlikely either one is leaving the other behind. The big problem they both have is they asked for the government to gate keep model releases and use cases, and their wish was granted.
That’s knocked them back 6 months already. Anthropic’s only frontier offering has been taken down.
Re: Apple Foundation Models
#135Earlier quoted context omitted.
I use both Claude and Codex and don’t see any meaningful difference between the two. My use case is modeling semi complex physical processes (energy and manufacturing) in code for simulations. I also have to do a good fair of automation via scripting in Python or PowerShell for manipulating data as well as legacy code analysis (C, Fortran, COBOL). Given I provide the models with the information and documentation they…
Did you find much of a difference between Fable and Opus?
I constantly hit safety blocks in Fable (I’m trying to write secure software, which is equivalent to finding security holes, so banned).
I didn’t use it on big enough tasks to notice any improvement.
I had been hitting plan limits pretty regularly, but fixed it by changing my workflow. That also increased the success rate of claude by an order of magnitude.
Re: Apple Foundation Models
#136Shared daemon is the only way this makes sense on-device. A 3B model at 4-bit is roughly 2GB - three apps loading their own copies would eat an 8GB phone.
Re: Apple Foundation Models
#137Earlier quoted context omitted.
Benedict Evans may be right after all; frontier models look more and more like telecom companies in the 90s. Billions and billions of investment in infrastructure while others further up the stack captured all the value.
In spite of their deeper pockets, massive datacenters, colosal amounts of user data, and hundreds of thousands of top developers, even Amazon, Meta, Microsoft, and Google are well behind. I think Evans is completely wrong. There are only 2 truly frontier models. (at least for now). And Anthropic seems to be leaving OpenAI behind so there might be only 1 in the near future. (which is scary/dangerous)
I think it's highly likely that there will remain one or two companies on the very bleeding edge of AI development for the foreseeable future.
But what I think a lot of people miss is that the market for the truly bleeding edge (developing bio-tech, building the most sophisticated software stacks (probably with a tilt towards simulation, GPU kernel optimization, etc)) is not the whole market.
There's a plethora of use-cases for models that are not on the bleeding edge. If I can solve my relatively simple problems with an off-the-shelf model for a minuscule fraction of the cost of the frontier, I'm going to.
Re: Apple Foundation Models
#138This is Apple commoditizing LLMs while keeping control of the UX. They are a hardware company and will keep selling the best machine for AI use. Well done.
Now we only need to commoditize the hardware.
Check out AMD’s offerings.
They’re typically a bit better on high TDP stuff, and a bit worse on low TDP. They mostly match in the middle. I have a $500 AMD NUC and a slightly older $2000 MBP. Inference throughput is within 2x.
The comparison is a little messy: AMD currently maxes out at 128GB of RAM vs Apple’s discontinued 512. Apple has nothing to rival the Steam Deck.
Re: Apple Foundation Models
#139Serious question: this looks like a thin library on an API. Why is it a big deal?
Re: Apple Foundation Models
#140Earlier quoted context omitted.
Benedict Evans may be right after all; frontier models look more and more like telecom companies in the 90s. Billions and billions of investment in infrastructure while others further up the stack captured all the value.
There will be frontier models that are non-commoditized, but they'll be kept guarded and hidden away, and you'll only get the final result, so that they can't be distilled and their harness can't be reverse engineered. They'll be billed like employees, rather than like a tool.
I doubt that. What stops the Chinese labs from figuring it out? It’s not like these models are fundamentally different from each other