Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Nativ: Run frontier open models locally on your Mac

blaizzy.github.io

Post reply on HN
Print view View original

31–40 of 144 posts

Re: Nativ: Run frontier open models locally on your Mac

#31

Post by sajithdilshan »

para_parolu wrote: post #27
dlandis wrote: post #12

Advice: remove all slop and fluff from the website such as "Everything you need. Nothing you don’t." Just state the information you want to communicate in the plainest and most straightforward way possible.

I don’t think anyone ever looked at result. Website if full of overflow bugs. It’s pure slop.

true, it doesn't render properly on mobile. Although I quite like the design. It's a new AI design guide (font/color choice) I haven't seen in in other AI designed pages

Re: Nativ: Run frontier open models locally on your Mac

#32

Post by sixtyj »

sajithdilshan wrote: post #29

Has anyone found a model that can run on a normal macbook? I have an M3 Pro with 18GB of memory and whenever I try to run even a basic model the fans goes off and the mac starts to get heated up and becomes so laggy.

Tbh, the only reason to run a model locally is when you want to be completely safe.

From productivity point of view, it doesn’t make sense to have any notebook running a local LLM.

We have one life. We should spend it wisely.

Re: Nativ: Run frontier open models locally on your Mac

#34

Post by calumcl »

asqueella wrote: post #6
iAMkenough wrote: post #3

How does this compare to LM Studio Bionic?

Bionic is an agent; this appears to be an open-source competitor to lmstudio. The initial commit is just a few hours ago though, so …

The maintainer works on mlx-vlm so he does have pedigree in the scene, I'm don't know if this is just going to end up as unmaintained slop. I haven't tried this but I would personally just recommend oMLX for a currently more complete and fleshed out package - loads of features, provides the same MLX support and changelogs + commits are actually detailed.

Re: Nativ: Run frontier open models locally on your Mac

#35

Post by erikkahler »

To clarify, this MIT-licensed app is from the very same dev, 'Prince Canuma', who maintains the popular MLX-VLM library (https://github.com/Blaizzy/mlx-vlm). MLX-VLM is a long-time dependency of the excellent LM Studio and others because it can provide faster inference on Apple devices than llama.cpp. Historically, MLX is a smaller community than CUDA, but has some of the fastest updates upon the release of new models, particularly in models with modalities beyond text-in, text-out (vision, STT, TTS, video gen). See also (https://github.com/Blaizzy/mlx-audio-swift). Would be totally unsurprised if those modalities and models get integrated into this UI.

Possibly vibe-coded landing page notwithstanding, the app is mostly written in Swift language. That suggests it will be easy to port this inference stack to iPad and iPhone.

Re: Nativ: Run frontier open models locally on your Mac

#36

Post by kzrdude »

D13Fd wrote: post #16

I'm surprised that their home page basically acts as if LM Studio and others don't already do this. It's not clear what the difference is from a glance. It also omits Open WebUI. I've been running Deepseek V4 Flash locally on my Macbook Pro for weeks using Open WebUI + DS4.

LM Studio seems to do the same thing, except that LM Studio is not open source. So they have a point, they do something more.

Re: Nativ: Run frontier open models locally on your Mac

#37

Post by mft_ »

sajithdilshan wrote: post #29

Has anyone found a model that can run on a normal macbook? I have an M3 Pro with 18GB of memory and whenever I try to run even a basic model the fans goes off and the mac starts to get heated up and becomes so laggy.

There is a Gemma 4 model with 12B parameters which might be worth trying. e.g. https://huggingface.co/mlx-community/gemma-4-12B-it-qat-4bit

That said, your computer will still get hot!

Re: Nativ: Run frontier open models locally on your Mac

#38

Post by kzrdude »

sajithdilshan wrote: post #29

Has anyone found a model that can run on a normal macbook? I have an M3 Pro with 18GB of memory and whenever I try to run even a basic model the fans goes off and the mac starts to get heated up and becomes so laggy.

If you go small enough it should be no problem. For example Gemma 4 E4B in Q6 or Q4 quantization should run well on your laptop. It shouldn't be too taxing, but would still want to eat 7-9 GB of VRAM or so.

Now that model is mostly useful for writing or chatting.

Re: Nativ: Run frontier open models locally on your Mac

#40

Post by dragonwriter »

moontear wrote: post #7

Is „frontier“ overused? I thought frontier models were the best-of-the-best such as Fable right now. I assume you can’t host these models yourself since you would need many GB of RAM and expensive GPU of is my thinking of „frontier models“ wrong?

The frontier is a multidimensional space defined by the “best” combination of traits a model can have in multiple dimensions: parameter size [smaller is better], various task metrics, and relative token generation speed on like hardware [faster is better], and active memory requirements [smaller is better] are all possible dimensions, and a model on the frontier of the current options space is one where getting better on one of those measures cannot be done without getting worse on at least one of the others.
Post reply on HN