Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Phi-4: Microsoft's Newest Small Language Model Specializing in Complex Reasoning

techcommunity.microsoft.com

Post reply on HN
Print view View original

11–20 of 148 posts

Re: Phi-4: Microsoft's Newest Small Language Model Specializing in Complex Reasoning

#11

Post by vulcanash999 »

lappa wrote: post #4
xeckr wrote: post #3

Looks like it punches way above its weight(s). How far are we from running a GPT-3/GPT-4 level LLM on regular consumer hardware, like a MacBook Pro?

It's easy to argue that Llama-3.3 8B performs better than GPT-3.5. Compare their benchmarks, and try the two side-by-side. Phi-4 is yet another step towards a small, open, GPT-4 level model. I think we're getting quite close. Check the benchmarks comparing to GPT-4o on the first page of their technical report if you haven't already https://arxiv.org/pdf/2412.08905

Did you mean Llama-3.1 8B? Llama 3.3 currently only has a 70B model as far as I’m aware.

Re: Phi-4: Microsoft's Newest Small Language Model Specializing in Complex Reasoning

#12

Post by thot_experiment »

For prompt adherence it still fails on tasks that Gemma2 27b nails every time. I haven't been impressed with any of the Phi family of models. The large context is very nice, though Gemma2 plays very well with self-extend.

Re: Phi-4: Microsoft's Newest Small Language Model Specializing in Complex Reasoning

#13

Post by BoorishBears »

simonw wrote: post #10
xeckr wrote: post #3

Looks like it punches way above its weight(s). How far are we from running a GPT-3/GPT-4 level LLM on regular consumer hardware, like a MacBook Pro?

We're there. Llama 3.3 70B is GPT-4 level and runs on my 64GB MacBook Pro: https://simonwillison.net/2024/Dec/9/llama-33-70b/ The Qwen2 models that run on my MacBook Pro are GPT-4 level too.

Saying these models are at GPT-4 level is setting anyone who doesn't place special value on the local aspect up for disappointment.

Some people do place value on running locally, and I'm not against then for it, but realistically no 70B class model has the amount of general knowledge or understanding of nuance as any recent GPT-4 checkpoint.

That being said these models are still very strong compared to what we had a year ago and capable of useful work

Re: Phi-4: Microsoft's Newest Small Language Model Specializing in Complex Reasoning

#14

Post by refulgentis »

xeckr wrote: post #3

Looks like it punches way above its weight(s). How far are we from running a GPT-3/GPT-4 level LLM on regular consumer hardware, like a MacBook Pro?

We're there, Llama 3.1 8B beats Gemini Advanced for $20/month. Telosnex with llama 3.1 8b GGUF from bartowski. https://telosnex.com/compare/ (How!? tl;dr: I assume Google is sandbagging and hasn't updated the underlying Gemini)

Re: Phi-4: Microsoft's Newest Small Language Model Specializing in Complex Reasoning

#15

Post by n144q »

simonw wrote: post #10
xeckr wrote: post #3

Looks like it punches way above its weight(s). How far are we from running a GPT-3/GPT-4 level LLM on regular consumer hardware, like a MacBook Pro?

We're there. Llama 3.3 70B is GPT-4 level and runs on my 64GB MacBook Pro: https://simonwillison.net/2024/Dec/9/llama-33-70b/ The Qwen2 models that run on my MacBook Pro are GPT-4 level too.

I wouldn't call 64GB MacBook Pro "regular consumer hardware".

Re: Phi-4: Microsoft's Newest Small Language Model Specializing in Complex Reasoning

#16

Post by syntaxing »

simonw wrote: post #9

The most interesting thing about this is the way it was trained using synthetic data, which is described in quite a bit of detail in the technical report: https://arxiv.org/abs/2412.08905 Microsoft haven't officially released the weights yet but there are unofficial GGUFs up on Hugging Face already. I tried this one: https://huggingface.co/matteogeniaccio/phi-4/tree/main I got it working with my LLM tool like this: l…

Wow, those responses are better than I expected. Part of me was expecting terrible responses since Phi-3 was amazing on paper too but terrible in practice.

Re: Phi-4: Microsoft's Newest Small Language Model Specializing in Complex Reasoning

#17

Post by simonw »

BoorishBears wrote: post #13
simonw wrote: post #10

Earlier quoted context omitted.

We're there. Llama 3.3 70B is GPT-4 level and runs on my 64GB MacBook Pro: https://simonwillison.net/2024/Dec/9/llama-33-70b/ The Qwen2 models that run on my MacBook Pro are GPT-4 level too.

Saying these models are at GPT-4 level is setting anyone who doesn't place special value on the local aspect up for disappointment. Some people do place value on running locally, and I'm not against then for it, but realistically no 70B class model has the amount of general knowledge or understanding of nuance as any recent GPT-4 checkpoint. That being said these models are still very strong compared to what we had a…

I said GPT-4, not GPT-4o. I'm talking about a model that feels equivalent to the GPT-4 we were using in March of 2023.

Re: Phi-4: Microsoft's Newest Small Language Model Specializing in Complex Reasoning

#18

Post by lifeisgood99 »

simonw wrote: post #9

The most interesting thing about this is the way it was trained using synthetic data, which is described in quite a bit of detail in the technical report: https://arxiv.org/abs/2412.08905 Microsoft haven't officially released the weights yet but there are unofficial GGUFs up on Hugging Face already. I tried this one: https://huggingface.co/matteogeniaccio/phi-4/tree/main I got it working with my LLM tool like this: l…

The SVG created for the first prompt is valid but is a garbage image.

Re: Phi-4: Microsoft's Newest Small Language Model Specializing in Complex Reasoning

#19

Post by jsheard »

n144q wrote: post #15
simonw wrote: post #10

Earlier quoted context omitted.

We're there. Llama 3.3 70B is GPT-4 level and runs on my 64GB MacBook Pro: https://simonwillison.net/2024/Dec/9/llama-33-70b/ The Qwen2 models that run on my MacBook Pro are GPT-4 level too.

I wouldn't call 64GB MacBook Pro "regular consumer hardware".

Yeah, a computer which starts at $3900 is really stretching that classification. Plus if you're that serious about local LLMs then you'd probably want the even bigger RAM option, which adds another $800...

Re: Phi-4: Microsoft's Newest Small Language Model Specializing in Complex Reasoning

#20

Post by simonw »

lifeisgood99 wrote: post #18
simonw wrote: post #9

The most interesting thing about this is the way it was trained using synthetic data, which is described in quite a bit of detail in the technical report: https://arxiv.org/abs/2412.08905 Microsoft haven't officially released the weights yet but there are unofficial GGUFs up on Hugging Face already. I tried this one: https://huggingface.co/matteogeniaccio/phi-4/tree/main I got it working with my LLM tool like this: l…

The SVG created for the first prompt is valid but is a garbage image.

Yeah, it didn't do very well on that one. The best I've had from a local model there was from QwQ: https://simonwillison.net/2024/Nov/27/qwq/
Post reply on HN