Is this included in the free AI tier for small developers? Big news if so
Apple Foundation Models
141–150 of 244 posts
Re: Apple Foundation Models
#142Earlier quoted context omitted.
Do you guys not have phones (with at least 1TB of storage)?
I have a Mac with 4TB of storage but it’s still annoying when every new AI app I try installs its own virtual environment with a fresh copy of Python, PyTorch, other duplicate libraries, and then models on top of that.
It’d be nice if there was a standard like ~/.local/llm/hugging-face-name.gguf or something.
Python heaviness is a more fundamental problem.
Re: Apple Foundation Models
#143Serious question: this looks like a thin library on an API. Why is it a big deal?
Shared daemon (as others pointed out), and, later shared revenue, probably with Apple receiving payments to ship ad-laden, “editorialized” models. Hopefully, it’ll go the other way, and Apple will subsidize high quality model training.
Re: Apple Foundation Models
#144Earlier quoted context omitted.
There will be frontier models that are non-commoditized, but they'll be kept guarded and hidden away, and you'll only get the final result, so that they can't be distilled and their harness can't be reverse engineered. They'll be billed like employees, rather than like a tool.
I doubt that. What stops the Chinese labs from figuring it out? It’s not like these models are fundamentally different from each other
If all you have is the starting point and the finishing point, the lack of the path taken from one point to another limits your ability to train models that can efficiently recreate the work, and increases its cost enough that it's possible the US labs can progress capabilities faster than Chinese labs can distill that behavior.
Re: Apple Foundation Models
#145Coding agent itself an imposed layer. Now they are adding one more layer? Many times I think of coding agent as the vendor supervisor from the body shops of the 90's who promise the customer everything under the sky and thrash the poor contractor to deliver. Coding agents consume 10x more tokens just like how body shops charged their customers vs how they paid the contractors. For a simple test, the same task that ma…
You wouldn't use this when building a coding agent.
How else will I run my coding agent on your Mac without having you download a second LLM and double your memory usage?
Re: Apple Foundation Models
#146Re: Apple Foundation Models
#147I'm not sure if I want to touch anything Anthropic anymore.
OpenAI is worse from a public policy standpoint, and apparently Fable was yanked at Amazon’s request.
Enough is enough. I’m seriously evaluating open models this week.
Re: Apple Foundation Models
#148Earlier quoted context omitted.
I have a Mac with 4TB of storage but it’s still annoying when every new AI app I try installs its own virtual environment with a fresh copy of Python, PyTorch, other duplicate libraries, and then models on top of that.
As an occasional python user I'm always amazed and frustrated that it seems that the only way to be able to use/build anything is to create a whole separate environment. And now given everybody now does this I guess the incentive to stop breaking stuff reduces even further. Might as well have static binaries.
That’s exactly how NPM works, and how Cargo works by default. You can make npm install stuff globally, but that’s not recommended except for things like CLI tooling. Cargo builds every project in its own separate targets/ directory unless you manually configure it to share that dir between builds. In both cases, the default is to isolate your current project from everything else on the system.
The main difference is that Python use to make you have to know that the virtualenv existed. Now `uv run` and `poetry run` abstract that away so you don’t have to interact with it if you don’t want to.
Re: Apple Foundation Models
#149> a Swift package that makes Claude available as a server-side language model in Apple's Foundation Models framework Ahh I was hoping for the opposite: all of the existing features of Claude Code but somehow running locally on my laptop's neural engine. A pipe dream on an M2 with 8 GB of RAM, but I had a flicker of hope there.
I've found most of the frontier coding models require somewhere between 300GB to 1TB to run with full capabilities.
The work on LLM in a Flash will probably help, and Apple's NVMe architecture is well suited to maximize throughput could allow their devices to work better on larger models than other vendors.
Re: Apple Foundation Models
#150Earlier quoted context omitted.
In spite of their deeper pockets, massive datacenters, colosal amounts of user data, and hundreds of thousands of top developers, even Amazon, Meta, Microsoft, and Google are well behind. I think Evans is completely wrong. There are only 2 truly frontier models. (at least for now). And Anthropic seems to be leaving OpenAI behind so there might be only 1 in the near future. (which is scary/dangerous)
>I think Evans is completely wrong. I wish there was a case where I find Evans is wrong. As far as my memory served me, I failed to record a single one. I disagree that Amazon, Meta, Microsoft, and Google are " well " behind. If anything the frontier model advantage seems to be at best 6 - 9 months. And that the Chinese model are all doing well. One of Steve Jobs's line, "It is a feature, not a product." Even if Appl…
Even their own employees get frustrated if they can't use Claude or Codex. 6-9 months is a big difference and I think it's closer to 9 than 6. And never mind the harness etc are also many months behind.