Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Phi-4: Microsoft's Newest Small Language Model Specializing in Complex Reasoning

techcommunity.microsoft.com

Post reply on HN
Print view View original

21–30 of 148 posts

Re: Phi-4: Microsoft's Newest Small Language Model Specializing in Complex Reasoning

#21

Post by refulgentis »

syntaxing wrote: post #16
simonw wrote: post #9

The most interesting thing about this is the way it was trained using synthetic data, which is described in quite a bit of detail in the technical report: https://arxiv.org/abs/2412.08905 Microsoft haven't officially released the weights yet but there are unofficial GGUFs up on Hugging Face already. I tried this one: https://huggingface.co/matteogeniaccio/phi-4/tree/main I got it working with my LLM tool like this: l…

Wow, those responses are better than I expected. Part of me was expecting terrible responses since Phi-3 was amazing on paper too but terrible in practice.

One of the funniest tech subplots in recent memory.

TL;DR it was nigh-impossible to get it to emit the proper "end of message" token. (IMHO the chat training was too rushed). So all the local LLM apps tried silently hacking around it. The funny thing to me was no one would say it out loud. Field isn't very consumer friendly, yet.

Re: Phi-4: Microsoft's Newest Small Language Model Specializing in Complex Reasoning

#22

Post by ant6n »

Terretta wrote: post #7
noman-land wrote: post #6

Earlier quoted context omitted.

The coolness of local LLMs is THE only reason I am sadly eyeing upgrading from M1 64GB to M4/5 128+GB.

Compare performance on various Macs here as it gets updated: https://github.com/ggerganov/llama.cpp/discussions/4167 OMM, Llama 3.3 70B runs at ~7 text generation tokens per second on Macbook Pro Max 128GB, while generating GPT-4 feeling text with more in depth responses and fewer bullets. Llama 3.3 70B also doesn't fight the system prompt, it leans in. Consider e.g. LM Studio (0.3.5 or newer) for a Metal (MLX) cente…

What if you have a Macbook Air with 16GB (the bechmarks dont seem to show memory).

Re: Phi-4: Microsoft's Newest Small Language Model Specializing in Complex Reasoning

#23

Post by russellbeattie »

n144q wrote: post #15
simonw wrote: post #10

Earlier quoted context omitted.

We're there. Llama 3.3 70B is GPT-4 level and runs on my 64GB MacBook Pro: https://simonwillison.net/2024/Dec/9/llama-33-70b/ The Qwen2 models that run on my MacBook Pro are GPT-4 level too.

I wouldn't call 64GB MacBook Pro "regular consumer hardware".

I have to disagree. I understand it's very expensive, but it's still a consumer product available to anyone with a credit card.

The comparison is between something you can buy off the shelf like a powerful Mac, vs something powered by a Grace Hopper CPU from Nvidia, which would require both lots of money and a business relationship.

Honestly, people pay $4k for nice TVs, refrigerators and even couches, and those are not professional tools by any stretch. If LLMs needed a $50k Mac Pro with maxed out everything, that might be different. But anything that's a laptop is definitely regular consumer hardware.

Re: Phi-4: Microsoft's Newest Small Language Model Specializing in Complex Reasoning

#24

Post by CuriousCosmic »

thot_experiment wrote: post #12

For prompt adherence it still fails on tasks that Gemma2 27b nails every time. I haven't been impressed with any of the Phi family of models. The large context is very nice, though Gemma2 plays very well with self-extend.

Yeah they mention this in the weaknesses section.

> While phi-4 demonstrates relatively strong performance in answering questions and performing reasoning tasks, it is less proficient at rigorously following detailed instructions, particularly those involving specific formatting requirements.

Re: Phi-4: Microsoft's Newest Small Language Model Specializing in Complex Reasoning

#25

Post by refulgentis »

lifeisgood99 wrote: post #18
simonw wrote: post #9

The most interesting thing about this is the way it was trained using synthetic data, which is described in quite a bit of detail in the technical report: https://arxiv.org/abs/2412.08905 Microsoft haven't officially released the weights yet but there are unofficial GGUFs up on Hugging Face already. I tried this one: https://huggingface.co/matteogeniaccio/phi-4/tree/main I got it working with my LLM tool like this: l…

The SVG created for the first prompt is valid but is a garbage image.

For context, pelican riding a bicycle: https://imgur.com/a/2nhm0XM

Copied SVG from gist into figma, added dark gray #444444 background, exported as PNG 1x.

Re: Phi-4: Microsoft's Newest Small Language Model Specializing in Complex Reasoning

#27

Post by bentcorner »

lifeisgood99 wrote: post #18
simonw wrote: post #9

The most interesting thing about this is the way it was trained using synthetic data, which is described in quite a bit of detail in the technical report: https://arxiv.org/abs/2412.08905 Microsoft haven't officially released the weights yet but there are unofficial GGUFs up on Hugging Face already. I tried this one: https://huggingface.co/matteogeniaccio/phi-4/tree/main I got it working with my LLM tool like this: l…

The SVG created for the first prompt is valid but is a garbage image.

In general I've had poor results with LLMs generating pictures using text instructions (in my case I've tried to get them to generate pictures using plots in KQL). They work but the pictures are very very basic.

I'd be interested for any LLM emitting any kind of text-to-picture instructions to get results that are beyond a kindergartner-cardboard-cutout levels of art.

Re: Phi-4: Microsoft's Newest Small Language Model Specializing in Complex Reasoning

#28

Post by simonw »

bentcorner wrote: post #27
lifeisgood99 wrote: post #18

Earlier quoted context omitted.

The SVG created for the first prompt is valid but is a garbage image.

In general I've had poor results with LLMs generating pictures using text instructions (in my case I've tried to get them to generate pictures using plots in KQL). They work but the pictures are very very basic. I'd be interested for any LLM emitting any kind of text-to-picture instructions to get results that are beyond a kindergartner-cardboard-cutout levels of art.

That's why I use the SVG pelican riding a bicycle thing as a benchmark: it's a deliberately absurd and extremely difficult task.

Re: Phi-4: Microsoft's Newest Small Language Model Specializing in Complex Reasoning

#29

Post by Havoc »

simonw wrote: post #9

The most interesting thing about this is the way it was trained using synthetic data, which is described in quite a bit of detail in the technical report: https://arxiv.org/abs/2412.08905 Microsoft haven't officially released the weights yet but there are unofficial GGUFs up on Hugging Face already. I tried this one: https://huggingface.co/matteogeniaccio/phi-4/tree/main I got it working with my LLM tool like this: l…

>Microsoft haven't officially released the weights

Thought it was official just not on huggingface but rather whatever azure competitor thing they're pushing?

Re: Phi-4: Microsoft's Newest Small Language Model Specializing in Complex Reasoning

#30

Post by bloomingkales »

xeckr wrote: post #3

Looks like it punches way above its weight(s). How far are we from running a GPT-3/GPT-4 level LLM on regular consumer hardware, like a MacBook Pro?

M4 Mac mini 16gb for $500. It's literally an inferencing block (small too, fits in my palm). I feel like the whole world needs one.
Post reply on HN