Live data from Hacker News

Phi-4: Microsoft's Newest Small Language Model Specializing in Complex Reasoning

techcommunity.microsoft.com

111–120 of 148 posts

Re: Phi-4: Microsoft's Newest Small Language Model Specializing in Complex Reasoning

#111
post #9

The most interesting thing about this is the way it was trained using synthetic data, which is described in quite a bit of detail in the technical report: https://arxiv.org/abs/2412.08905 Microsoft haven't officially released the weights yet but there are unofficial GGUFs up on Hugging Face already. I tried this one: https://huggingface.co/matteogeniaccio/phi-4/tree/main I got it working with my LLM tool like this: l…

When working with GGUF what chat templates do you use? Pretty much every gguf I've imported into ollama has given me garbage response. Converting the tokenizer json has yielded mixed results. For example how do you handle the phi-4 models gguf chat template?

I use whatever what template is baked into the GGUF file.

You can click on the little info icon on Hugging Face to see that directly.

For https://huggingface.co/matteogeniaccio/phi-4/tree/main?show_... that's this:

  {% for message in messages %}{% if
  (message['role'] == 'system')
  %}{{'system' +
  message['content'] + ''}}{%
  elif (message['role'] == 'user')
  %}{{'user' +
  message['content'] +
  'assistant'}}{%
  elif (message['role'] == 'assistant')
  %}{{message['content'] + ''}}{%
  endif %}{% endfor %}

Re: Phi-4: Microsoft's Newest Small Language Model Specializing in Complex Reasoning

#112
post #68

Earlier quoted context omitted.

A Mac with 16GB RAM can run qwen 7b, gemma 9b and similar models that are somewhere between GPT3.5 and GPT4. Quite impressive.

on what metric? Why would OpenAI bother serving GPT4 if customers would be just as happy with a tiny 9B model?

Don't confuse GPT-4 and GPT-4o.

GPT-4o is a much better experience than the smaller local models. You can see that in the lmarena benchmarks or from trying them out yourself.

Re: Phi-4: Microsoft's Newest Small Language Model Specializing in Complex Reasoning

#114
post #89
post #9

The most interesting thing about this is the way it was trained using synthetic data, which is described in quite a bit of detail in the technical report: https://arxiv.org/abs/2412.08905 Microsoft haven't officially released the weights yet but there are unofficial GGUFs up on Hugging Face already. I tried this one: https://huggingface.co/matteogeniaccio/phi-4/tree/main I got it working with my LLM tool like this: l…

> it was trained using synthetic data Is this not supposed to cause Model collapse?

As someone who's a completely layman: I wonder if the results of model collapse are no worse than, say, sufficiently complex symbolic AI (modulo consistency and fidelity?)

Re: Phi-4: Microsoft's Newest Small Language Model Specializing in Complex Reasoning

#115
post #102

Earlier quoted context omitted.

This "draw pelican riding on bicycle" is quite deep if you think about it. Phi is all about synthetic training and prompt -> svg -> render -> evaluate image -> feedback loop feels like ideal fit for synthetic learning. You can push it quite far with stuff like basic 2d physics etc with plotting scene after N seconds or optics/rays, magnetic force etc. SVG as LLM window to physical world.

> SVG as LLM window to physical world. What? let’s try not to go full forehead into hype. SVGs would be an awfully poor analogy for the physical world…

SVGs themselves are just an image format; but because of their vector nature, they could easily be mapped onto values from a simulation in a physics engine — at least, in the game physics sense of the word, rods and springs etc., as a fluid simulation is clearly a better map to raster formats.

If that physics engine were itself a good model for the real world, then you could do simulated evolution to get an end result that is at least as functional as a bike (though perhaps it wouldn't look like a traditional bike) even if the only values available to the LLM were the gross characteristics like overall dimensions and mass.

But I'd say the chance of getting a pelican SVG out of a model like this is mostly related to lots of text describing the anatomy of pelicans, and it would not gain anything from synthetic data.

Re: Phi-4: Microsoft's Newest Small Language Model Specializing in Complex Reasoning

#116
post #115
post #102

Earlier quoted context omitted.

> SVG as LLM window to physical world. What? let’s try not to go full forehead into hype. SVGs would be an awfully poor analogy for the physical world…

SVGs themselves are just an image format; but because of their vector nature, they could easily be mapped onto values from a simulation in a physics engine — at least, in the game physics sense of the word, rods and springs etc., as a fluid simulation is clearly a better map to raster formats. If that physics engine were itself a good model for the real world, then you could do simulated evolution to get an end resul…

> but because of their vector nature, they could easily be mapped onto values from a simulation in a physics engine.

I don’t think the fact that the images are described with vectors magically makes it better for representing physics than any other image representation. Maybe less so, since there will be so much textual information not related to the physical properties of the object.

What about them makes it easier to map to physics than an AABB?

For soft body physics, im pretty sure a simpler sort of distance field representation would even be better. (I’m not as familiar with soft body as rigid body)

Re: Phi-4: Microsoft's Newest Small Language Model Specializing in Complex Reasoning

#117

So we moved from "reasoning" to "complex reasoning". I wonder what will be next month's buzzphrase.

> So we moved from "reasoning" to "complex reasoning". Only from the perspective of those still complaining about the use of the term "reasoning", who now find themselves left behind as the world has moved on. For everyone else, the phrasing change perfectly fits the technological change.

Reasoning basically means multi-step prediction, but to be general the reasoner also needs to be able to:

1) Realize when it's reached an impasse, then backtrack and explore alternatives

2) Recognize when no further progress towards the goal appears possible, and switch from exploiting existing knowledge to exploring/acquiring new knowledge to attempt to proceed. An LLM has limited agency, but could for example ask a question or do a web search.

In either case, prediction failure needs to be treated as a learning signal so the same mistake isn't repeated, and when new knowledge is acquired that needs to be remembered. In both cases this learning would need to persist beyond the current context in order to be something that the LLM can build on in the future - e.g. to acquire a job skill that may take a lot of experience/experimentation to master.

It doesn't matter what you call it (basic or advanced), but it seems that current attempts at adding reasoning to LLMs (e.g. GPT-o1) are based around 1), a search-like strategy, and learning is in-context and ephemeral. General animal-like reasoning needs to also support 2) - resolving impasses by targeted new knowledge acquisition (and/or just curiosity-driven experimentation), as well as continual learning.

Re: Phi-4: Microsoft's Newest Small Language Model Specializing in Complex Reasoning

#119

So we moved from "reasoning" to "complex reasoning". I wonder what will be next month's buzzphrase.

If you graded humanity on their reasoning ability, I wonder where these models would score? I think once they get to about the 85th percentile, we could upgrade the phrase to advanced reasoning. I'm roughly equating it with the percentage of the US population with at least a master's degree.

All current LLMs openly make simple mistakes that are completely incompatible with true "reasoning" (in the sense any human would have used that term years ago).

I feel like I'm taking crazy pills sometimes.

Re: Phi-4: Microsoft's Newest Small Language Model Specializing in Complex Reasoning

#120
post #15
post #10

Earlier quoted context omitted.

We're there. Llama 3.3 70B is GPT-4 level and runs on my 64GB MacBook Pro: https://simonwillison.net/2024/Dec/9/llama-33-70b/ The Qwen2 models that run on my MacBook Pro are GPT-4 level too.

I wouldn't call 64GB MacBook Pro "regular consumer hardware".

[deleted]
Post reply on HN