Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ollama is now powered by MLX on Apple Silicon in preview

ollama.com

Post reply on HN
Print view View original

201–210 of 384 posts

Re: Ollama is now powered by MLX on Apple Silicon in preview

#201

Post by RALaBarge »

aurareturn wrote: post #62
pezgrande wrote: post #35

Earlier quoted context omitted.

> have to release free open source models because they distill from OpenAI and Anthropic They dont really have to though, they just need to be good enough and cheaper (even if distilled). That being said, it is true they are gaining a lot of visibility (specially Qwen) because of being open-source(weight). Hardware-wise they seem they will catch-up in 3-5 years (Nvidia is kind of irrelevant, what matters is the node)…

I highly doubt they can catch up in 3-5 years to Nvidia. Chips take about 3 years to design. Do you think China will have Feymann-level AI systems in 3 years? I think in 3 years, they'll have H200-equivalent at home.

You must have an inside line on information for 'China' -- those are bold predictions!

Re: Ollama is now powered by MLX on Apple Silicon in preview

#202

Post by beepbooptheory »

franze wrote: post #125
LeoDaVibeci wrote: post #116

Earlier quoted context omitted.

Honestly I can't believe Apple put that foundation model product out the door. I was so excited about it, but when I tried it, it was such a disappointment. Glad to hear you calling that out so I know it wasn't just me. Looks like they have pivoted completely over to Gemini, thank god.

yeah, it is super limited but also you can now do cmd(){ local x c r a; while [[ $1 == -* ]]; do case $1 in -x)x=1;shift;; -c)c=1;shift;; *)break;; esac; done; r=$(apfel -q -s 'Output only a shell command.' "$*" | sed '/^```/d;/^#/d;s/^[[:space:]]*//;/^$/d' | head -1); [[ $r ]] || { echo "no command generated"; return 1; }; printf '\e[32m$\e[0m %s\n' "$r"; [[ $c ]] && printf %s "$r" | pbcopy && echo "(copied)"; [[ $x…

What is the AI doing here? Or is this just like being cheeky?

Re: Ollama is now powered by MLX on Apple Silicon in preview

#204

Post by adam_patarino »

babblingfish wrote: post #2

LLMs on device is the future. It's more secure and solves the problem of too much demand for inference compared to data center supply, it also would use less electricity. It's just a matter of getting the performance good enough. Most users don't need frontier model performance.

[flagged]

Re: Ollama is now powered by MLX on Apple Silicon in preview

#205

Post by PhilipRoman »

Barbing wrote: post #64
asutekku wrote: post #48

Earlier quoted context omitted.

Frontier model has much better knowledge and they usually hallucinate less. It's not about the coding capabilities, it's about how much you can trust the model.

re: trust- Have you tried the free version of ChatGPT? It is positively appalling. It’s like GPT 3.5 but prompted to write three times as much as necessary to seem useful. I wonder how many people have embarrassed themselves, lost their jobs, and been critically misinformed. All easy with state-of-the-art models but seemingly a guarantee with the bottom sub-slop tier. Is the average person just talking to it about th…

I use the free version of ChatGPT (without logging in) when I need some one-off question without a huge context. Real world prompt:

  "when hostapd initializes 80211 iface over nl80211, what attributes correspond to selected standard version like ax or be?"
It works fine, avoids falling into trap due to misleading question. Probably works even better for more popular technologies. Yeah, it has higher failure rates but it's not a dealbreaker for non-autonomous use cases.

Re: Ollama is now powered by MLX on Apple Silicon in preview

#206

Post by spiderfarmer »

aurareturn wrote: post #60
seanmcdirmid wrote: post #41

Earlier quoted context omitted.

About 2.5 decades from the start of the JVs, but they did it. Semiconductors and jet turbines are really the last two tech trees that China has yet to master.

Right. When I said "they'll always be behind", I meant in the next 5-10 years. They're gated by EUV tech. And once they have EUV tech, they need to scale up chip manufacturing.

You will always be wrong.

Re: Ollama is now powered by MLX on Apple Silicon in preview

#208

Post by eeixlk »

babblingfish wrote: post #2

LLMs on device is the future. It's more secure and solves the problem of too much demand for inference compared to data center supply, it also would use less electricity. It's just a matter of getting the performance good enough. Most users don't need frontier model performance.

Obviously apple would prefer this. It would boost demand for more powerful and expensive devices, and align with their privacy marketing. But they have massively fumbled with siri for a long time and then missed huge deadlines with ai promises. Despite having billions, they have shown no competency in delivering services or accurately marketing what to expect from ai features.

Re: Ollama is now powered by MLX on Apple Silicon in preview

#209

Post by Aurornis »

noritaka88 wrote: post #207

[flagged]

> The local inference story is getting real — I've been running 9 autonomous agents on a Mac mini (Haiku via API, not local yet) and the biggest bottleneck isn't the model, it's the coordination layer between agents. Identity, settlement, who-did-what.

What does this comment have to do with MLX or the story?

Actually, is this just an LLM posting too? This has em-dashes, “it’s not this, it’s that”, and a rule of three statement at the end.

EDIT: Account is posting multiple long comments on different threads only 1-2 minutes apart. This is a bot.

Re: Ollama is now powered by MLX on Apple Silicon in preview

#210

Post by dgacmu »

beepbooptheory wrote: post #202
franze wrote: post #125

Earlier quoted context omitted.

yeah, it is super limited but also you can now do cmd(){ local x c r a; while [[ $1 == -* ]]; do case $1 in -x)x=1;shift;; -c)c=1;shift;; *)break;; esac; done; r=$(apfel -q -s 'Output only a shell command.' "$*" | sed '/^```/d;/^#/d;s/^[[:space:]]*//;/^$/d' | head -1); [[ $r ]] || { echo "no command generated"; return 1; }; printf '\e[32m$\e[0m %s\n' "$r"; [[ $c ]] && printf %s "$r" | pbcopy && echo "(copied)"; [[ $x…

What is the AI doing here? Or is this just like being cheeky?

The pile of shell and sed is cleaning up the ai output and then running it in the shell.

The instruction to the AI was to create _a_ shell command. So it's a random shell command generator (maybe).

Post reply on HN