I've just tried to make it run something, and I just could not force to include the python code inside ``` ``` quotation marks. It always wants to put word python after three quotes, like this:
```python
.. code..
```
I wonder if that's the result of training.
(I use the LLM output to then run the resulting code)
Phi 4 available on Ollama
131–138 of 138 posts
Re: Phi 4 available on Ollama
#132Earlier quoted context omitted.
> in the consumer conversational AI space. The "consumer conversational AI space" only exists right now as a novelty, not a long-term market segment. In the not too distant future that space will be covered for most users for free by their hardware manufacturers, and the number of people willing to pay a monthly subscription to a third party will drop even further than it already has.
I think it will be at least a few years until your average Joe can run a speech to speech model on their phone.
I didn't say anything about running locally. Siri and Google Assistant (Gemini) are what I had in mind: assistants bundled with the phone will remove the need to pay for ChatGPT.
Re: Phi 4 available on Ollama
#133Does this have an "instruct" version? Or it's already sort-of like that, as it was trained more on Q&A scenarios?
Re: Phi 4 available on Ollama
#134Over the holidays, we published a post[1] on using high-precision few-shot examples to get `gpt-4o-mini` to perform similar to `gpt-4o`. I just re-ran that same experiment, but swapped out `gpt-4o-mini` with `phi-4`. `phi-4` really blew me away in terms of learning from few-shots. It measured as being 97% consistent with `gpt-4o` when using high-precision few-shots! Without the few-shots, it was only 37%. That's a hu…
m ∈ ℤ is the threshold for determining high or low novelty
search(T,θ,m) retrieves the first m historical tasks that are semantically similar above the θ threshold
Are both m's here the same or different numbers? I found this a bit confusing
Re: Phi 4 available on Ollama
#135Earlier quoted context omitted.
This is really nice. I loved the detailed process and I'm definitely gonna use it. One nit though: I didn't understand what the graphs mean, maybe you should add the axes names.
Thanks! Great suggestion for improving the graphs – I just updated the post with axis labels.
As a bit minor criticism "Tasks Completed (Time)" is hard to evaluate without the time intervals or units. I'm not sure if it should just be "Time"?
Re: Phi 4 available on Ollama
#136Over the holidays, we published a post[1] on using high-precision few-shot examples to get `gpt-4o-mini` to perform similar to `gpt-4o`. I just re-ran that same experiment, but swapped out `gpt-4o-mini` with `phi-4`. `phi-4` really blew me away in terms of learning from few-shots. It measured as being 97% consistent with `gpt-4o` when using high-precision few-shots! Without the few-shots, it was only 37%. That's a hu…
m ∈ ℤ is the threshold for determining high or low novelty search(T,θ,m) retrieves the first m historical tasks that are semantically similar above the θ threshold Are both m's here the same or different numbers? I found this a bit confusing
In our case, yes we treat them the same. Though it might be interesting to decouple them.
You could, for example, include all few-shots that meet the similarity threshold, but you’ll use more tokens for (I assume) marginal gain. Definitely worth a try though.
Re: Phi 4 available on Ollama
#137Earlier quoted context omitted.
I am blown away: a year ago I bought a M2 32G Mac to run local models. It seems like what I can run locally now just one year later is 10x more useful for NLP, data wrangling, RAG, experimenting with agents, etc. BTW, a few days ago I published a book on using Ollama. Here is a link to read it online https://leanpub.com/ollama/read
Which models do you recommend for that amount of memory?
For reasoning: qwq:latest (19G file)
For coding: qwen2.5-coder:14b (9G file)
Misc. experiments, runs fast: llama3.2:latest )2 G file)
Re: Phi 4 available on Ollama
#138Can this run on a macbook m1? What is the performance like? Or would I need an m3? Thanks
Yeah as long as it has 16GB+ RAM. I've got a newer cpu and it's very fast, so I expect on an M1 it would be at least bearable.
It's good enough for me on an M1, 16Gb, and slow but good enough as a background job on my older intel mbp with 16Gb. I somehow expected it to not work on intel macs at all, so that's a freebie.