The bigger news is that Phi-3-Small, Phi-3-Medium and Phi-3-Vision were finally released
"finally"!? Vision wasn't even on the table until today! And they're clearly rushed and fundamentally broken for chat use cases.
Microsoft Phi-3 Cookbook
11–20 of 59 posts
Re: Microsoft Phi-3 Cookbook
#12The initial model release had a terrible, frequent, issue with emitting the wrong "end of message" token, or never emitting one.[1] That is a very serious issue that breaks chat. The ones from today still have this issue.[2] Beyond that, they've been pushing new ONNX features enabling LLMs via Phi for about a month now. The ONNX runtime that supports it still isn't out, much less the downstream integration of it into…
in the screenshot you shared in the twitter link [2], the model appears to do everything correctly - it terminated its message with which is correct according to the Phi-3 prompt format. It seems whatever environment you are hosting it in does not understand that should be considered the EOS string, and not ??
Re: Microsoft Phi-3 Cookbook
#13The initial model release had a terrible, frequent, issue with emitting the wrong "end of message" token, or never emitting one.[1] That is a very serious issue that breaks chat. The ones from today still have this issue.[2] Beyond that, they've been pushing new ONNX features enabling LLMs via Phi for about a month now. The ONNX runtime that supports it still isn't out, much less the downstream integration of it into…
in the screenshot you shared in the twitter link [2], the model appears to do everything correctly - it terminated its message with which is correct according to the Phi-3 prompt format. It seems whatever environment you are hosting it in does not understand that should be considered the EOS string, and not ??
Good call, it's an HF space for Phi Vision, maybe someone jumped the gun / didn't set stuff up properly, or it's splitting into multiple tokens
Re: Microsoft Phi-3 Cookbook
#14You can interact with the new Phi-3 vision model on this page (no login required): https://ai.azure.com/explore/models/Phi-3-vision-128k-instru...
I get a "the request was blocked" error.
Re: Microsoft Phi-3 Cookbook
#15"We are introducing Phi Silica which is built from the Phi series of models and is designed specifically for the NPUs in Copilot+ PCs. Windows is the first platform to have a state-of-the-art small language model (SLM) custom built for the NPU and shipping inbox. Phi Silica API along with OCR, Studio Effects, Live Captions, Recall User Activity APIs will be available in Windows Copilot Library in June. More APIs like…
For the price of these new laptops one can already buy a MacBook with a general purpose GPU that is more than capable of running these models. One can buy a windows laptop with a dedicated graphics card that can also run these models. Perhaps it would be interesting if the price was substantially lower.
These new laptops are half the price of macbooks with the same ram and storage
Re: Microsoft Phi-3 Cookbook
#16Earlier quoted context omitted.
For the price of these new laptops one can already buy a MacBook with a general purpose GPU that is more than capable of running these models. One can buy a windows laptop with a dedicated graphics card that can also run these models. Perhaps it would be interesting if the price was substantially lower.
These new laptops are half the price of macbooks with the same ram and storage
In Australia they are the same price ($1.9k AUD vs $1.6k for MacBook air). How much where you're at?
Re: Microsoft Phi-3 Cookbook
#17The initial model release had a terrible, frequent, issue with emitting the wrong "end of message" token, or never emitting one.[1] That is a very serious issue that breaks chat. The ones from today still have this issue.[2] Beyond that, they've been pushing new ONNX features enabling LLMs via Phi for about a month now. The ONNX runtime that supports it still isn't out, much less the downstream integration of it into…
FYI, there's a recipe for running Phi-3 under ONNX on iOS in the linked repository https://github.com/microsoft/Phi-3CookBook/blob/main/md/03.I...
Re: Microsoft Phi-3 Cookbook
#18Re: Microsoft Phi-3 Cookbook
#19Wow, actually this cookbook is really bad? I expected something like the OpenAI or Anthropic cookbooks, but this seems to be some AI generated low-quality content without any code examples or interesting examples?
The Phi-3 models are great though, especially the vision model has great potential for low latency applications (like robotics?)...
Re: Microsoft Phi-3 Cookbook
#20Earlier quoted context omitted.
For the price of these new laptops one can already buy a MacBook with a general purpose GPU that is more than capable of running these models. One can buy a windows laptop with a dedicated graphics card that can also run these models. Perhaps it would be interesting if the price was substantially lower.
These new laptops are half the price of macbooks with the same ram and storage
In which configs and which geographic locations? In the US, pricing for equivalent Surface and Mac hardware configurations looks the same, to me.
And that's if the Snapdragon X Elite is actually on par with the Apple M3, like Microsoft claims.
Earlier X Elite benchmarks[1][2] showed that it was behind the M3, but hopefully Qualcomm have made some solid performance changes since then. Competition is good.
1. https://www.xda-developers.com/snapdragon-x-elite-benchmarks... Note: the link is comparing vs the M2
2. https://www.tomshardware.com/pc-components/cpus/early-snapdr...