Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Phi 4 available on Ollama

ollama.com

Post reply on HN
Print view View original

1–10 of 138 posts

Re: Phi 4 available on Ollama

#4

Post by compumetrika »

andhuman wrote: post #3

I’ve seen on the localllama subreddit that some GGUFs have bugs in them. The one recommended was by unsloth. However, I don’t know how the Ollama GGUF holds up.

Ollama can pull directly from HF, you just provide the URL and add to the end :Q8_0 (or whatever) to specify your desired quant. Bonus: use the short form url of `hf` instead of `huggingface` to shorten the model name a little in the ollama list table.

Edit: so for example of you want the unsloth "debugged" version of Phi4, you would run:

`$ollama pull hf.co/unsloth/phi-4-GGUF:Q8_0`

(check on the right side of the hf.co/unsloth/phi-4-GGUF page for the available quants)

Re: Phi 4 available on Ollama

#5

Post by magicalhippo »

andhuman wrote: post #3

I’ve seen on the localllama subreddit that some GGUFs have bugs in them. The one recommended was by unsloth. However, I don’t know how the Ollama GGUF holds up.

Here's[1] a recent submission on that.

[1]: https://news.ycombinator.com/item?id=42660335 Phi-4 Bug Fixes

Re: Phi 4 available on Ollama

#6

Post by mythz »

Was disappointed in all the Phi models before this, whose benchmark results scored way better than it worked in practice, but I've been really impressed with how good Phi-4 is at just 14B. We've run it against the top 1000 most popular StackOverflow questions and it came up 3rd beating out GPT-4 and Sonnet 3.5 in our benchmarks, only behind DeepSeek v3 and WizardLM 8x22B [1]. We're using Mixtral 8x7B to grade the quality of the answers which could explain how WizardLM (based on Mixtral 8x22B) took 2nd Place.

Unfortunately I'm only getting 6 tok/s on NVidia A4000 so it's still not great for real-time queries, but luckily now that it's MIT licensed it's available on OpenRouter [2] for a great price of $0.07/$0.14M at a fast 78 tok/s.

Because it yields better results and we're able to self-host Phi-4 for free, we've replaced Mistral NeMo with it in our default models for answering new questions [3].

[1] https://pvq.app/leaderboard

[2] https://openrouter.ai/microsoft/phi-4

[3] https://pvq.app/questions/ask

Re: Phi 4 available on Ollama

#7

Post by [deleted] »

gnabgib wrote: post #2

Related Phi-4: Microsoft's Newest Small Language Model Specializing in Complex Reasoning (439 points, 24 days ago, 144 comments) https://news.ycombinator.com/item?id=42405323 Also on hugging face https://huggingface.co/microsoft/phi-4

[deleted]

Re: Phi 4 available on Ollama

#8

Post by KTibow »

mythz wrote: post #6

Was disappointed in all the Phi models before this, whose benchmark results scored way better than it worked in practice, but I've been really impressed with how good Phi-4 is at just 14B. We've run it against the top 1000 most popular StackOverflow questions and it came up 3rd beating out GPT-4 and Sonnet 3.5 in our benchmarks, only behind DeepSeek v3 and WizardLM 8x22B [1]. We're using Mixtral 8x7B to grade the qua…

Interesting eval but my first reaction is "using Mixtral as a judge doesn't sound like a good idea". Have you tested how different its results are from GPT-4 as a judge (on a small scale) or how stuff like style and order can affect its judgements?

Edit: they have a blog post https://pvq.app/posts/individual-voting-comparison although it could go deeper

Re: Phi 4 available on Ollama

#9

Post by crorella »

It’s odd that MS is releasing models they are competitors to OA. This reinforce the idea that there is no real strategic advantage in owning a model. I think the strategy is now offer cheap and performant infra to run the models.
Post reply on HN