To clarify, this MIT-licensed app is from the very same dev, 'Prince Canuma', who maintains the popular MLX-VLM library ( https://github.com/Blaizzy/mlx-vlm ). MLX-VLM is a long-time dependency of the excellent LM Studio and others because it can provide faster inference on Apple devices than llama.cpp. Historically, MLX is a smaller community than CUDA, but has some of the fastest updates upon the release of new mod…
Thanks for that. My first question was “What does this do that Unsloth doesn’t?”
Ironic that the app is named Nativ(e) and yet bundles a full Python runtime. Nonetheless, still less bloated than LM Studio, which bundles a full Python runtime and electron.js (which in turn bundles a whole browser runtime).
Is Gemma 4 E2B actually "usable"?
I've been running Gemma 4 12B and it handles everything very well!
But the second I've moved down to E4B it's been unable to perform the simplest of tasks.
So I can't even imagine how E2B would do...
Or am I doing something wrong?
You’d be surprised how hard this actually is. I spent 3 days iterating on a marketing site, where I had very explicit / “well written” copy, and it would just repeatedly rewrite it back to the most awful slop. Over and over again! Ended up adding various AGENTS rules telling it to leave the copy alone
Just...write it yourself.
I did. And then it rewrote it. Over and over again. And I kept restoring it. And it kept taking agency to change it to some other neutral slop.
Has anyone found a model that can run on a normal macbook? I have an M3 Pro with 18GB of memory and whenever I try to run even a basic model the fans goes off and the mac starts to get heated up and becomes so laggy.
There is a Gemma 4 model with 12B parameters which might be worth trying. e.g. https://huggingface.co/mlx-community/gemma-4-12B-it-qat-4bit That said, your computer will still get hot!
I just run Gemma4 12B MLX via Ollama and it's been doing fantastic work!
Ironic that the app is named Nativ(e) and yet bundles a full Python runtime. Nonetheless, still less bloated than LM Studio, which bundles a full Python runtime and electron.js (which in turn bundles a whole browser runtime).
A lot of macOS apps are statically-linked, even interpreted programs. It's still a native app for going that route.
Genuinely curious: what are people using these smaller local models for? They are getting decently capable, but they are still small enough that I don't trust them for "real" work outside of a handful of fun toy projects.
Are people actually using them in coding agents? Or are they mostly using them for other things?
Genuinely curious: what are people using these smaller local models for? They are getting decently capable, but they are still small enough that I don't trust them for "real" work outside of a handful of fun toy projects. Are people actually using them in coding agents? Or are they mostly using them for other things?
They're great at helping me look up web dev stuff when I don't have internet access.
Bionic is an agent; this appears to be an open-source competitor to lmstudio. The initial commit is just a few hours ago though, so …
The maintainer works on mlx-vlm so he does have pedigree in the scene, I'm don't know if this is just going to end up as unmaintained slop. I haven't tried this but I would personally just recommend oMLX for a currently more complete and fleshed out package - loads of features, provides the same MLX support and changelogs + commits are actually detailed.
I use oMLX and I'm tentatively going to be giving this a shot. oMLX keeps driving me up a wall with odd papercuts, bugs, and silent failures and fallbacks that are only visible buried deep inside logs when they should be announced out loud.
The maintainer of mlx-vlm being behind this as well is the main thing kicking me over into trying it, even if it is incredibly young. I'm confused and unenthused to see it chomping on a whole GB of disk, but the Swift makes it feel much more refined even if it's not yet as feature rich. It automatically picked up the existing models I was using with mlx_vm directly, which was nifty.
Genuinely curious: what are people using these smaller local models for? They are getting decently capable, but they are still small enough that I don't trust them for "real" work outside of a handful of fun toy projects. Are people actually using them in coding agents? Or are they mostly using them for other things?
We've shipped some code generated by Qwen3.6 27B to production (under OpenCode). It lacks the breadth of knowledge of models like Opus, but if a change is fully inferable from the prompt and the surrounding code, it works very well. It won't be able to write something from scratch that requires niche knowledge (say, a performant inference engine tailored to Blackwell GPUs), but if it's just a PR adding a new use case to an existing project (which is usually just "load from the DB, do some invariant checks, modify the entities, store them back"), it works as well as Sonnet (provided you have the correct configuration, like recommended temperature and top-p settings, the model isn't over-quantized, you have at least 150k tokens of context available, etc.).