Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Phi 4 available on Ollama

ollama.com

Post reply on HN
Print view View original

51–60 of 138 posts

Re: Phi 4 available on Ollama

#51

Post by chefandy »

emidoots wrote: post #50
chefandy wrote: post #45

Earlier quoted context omitted.

> I have no idea what game Meta is playing Based on their business moves in recent history, I’d guess most of them are playing Farmville.

Meta's entire business model is to own users and their content. Whether it be Facebook, Instagram, Threads, Messenger, WhatsApp, etc. their focus is to acquire users, keep them in their platforms, and own their content - because /human attention is fundamentally valuable/. Meta owns 40% of the most popular social media platforms today, but their attention economies face great threats: YouTube, TikTok, Telegram, WeCha…

Word. I was mostly just making a joke about FarmVille— the classic engagement-vampire facebook game.

Re: Phi 4 available on Ollama

#52

Post by sgk284 »

yard2010 wrote: post #46
sgk284 wrote: post #27

Over the holidays, we published a post[1] on using high-precision few-shot examples to get `gpt-4o-mini` to perform similar to `gpt-4o`. I just re-ran that same experiment, but swapped out `gpt-4o-mini` with `phi-4`. `phi-4` really blew me away in terms of learning from few-shots. It measured as being 97% consistent with `gpt-4o` when using high-precision few-shots! Without the few-shots, it was only 37%. That's a hu…

This is really nice. I loved the detailed process and I'm definitely gonna use it. One nit though: I didn't understand what the graphs mean, maybe you should add the axes names.

Thanks! Great suggestion for improving the graphs – I just updated the post with axis labels.

Re: Phi 4 available on Ollama

#53

Post by lmeyerov »

sgk284 wrote: post #27

Over the holidays, we published a post[1] on using high-precision few-shot examples to get `gpt-4o-mini` to perform similar to `gpt-4o`. I just re-ran that same experiment, but swapped out `gpt-4o-mini` with `phi-4`. `phi-4` really blew me away in terms of learning from few-shots. It measured as being 97% consistent with `gpt-4o` when using high-precision few-shots! Without the few-shots, it was only 37%. That's a hu…

I like the direction, but have a pretty different experience in practice. This spans legal analytics, social media analytics, code synthesis, news analysis, cyber security LLMs, etc:

1. The only ultimate absolute quality metric I saw in that blogpost afaict was expert agreement... at 90%. All of our customers would fire us at that level across all of the diff b2b domains we work in. I'm surprised 90% is considered acceptable quality in a paying business context like retail.

2. Gpt-4o-mini is great. I find we can get, for these kind of simple tasks you describe, gpt-4o-mini to achieve about 95-98% agreement with gpt-4o by iteratively manually improving prompts over increasingly large synthetic evals. Given data and a good dev, we do this basically same-day for a lot of simple tasks, which is astounding.

I do expect automatic prompt optimizers to win here long-term, and keep hopefully revisiting dspy et al. For now, they fail over standard prompt engineering. Likewise, I do believe in example learning over time for areas like personalization.... but doing semantic search recall of high-rated answers was a V1 thing we had to rethink due to too many issues.

Re: Phi 4 available on Ollama

#54

Post by miki123211 »

emidoots wrote: post #50
chefandy wrote: post #45

Earlier quoted context omitted.

> I have no idea what game Meta is playing Based on their business moves in recent history, I’d guess most of them are playing Farmville.

Meta's entire business model is to own users and their content. Whether it be Facebook, Instagram, Threads, Messenger, WhatsApp, etc. their focus is to acquire users, keep them in their platforms, and own their content - because /human attention is fundamentally valuable/. Meta owns 40% of the most popular social media platforms today, but their attention economies face great threats: YouTube, TikTok, Telegram, WeCha…

Or, to put it another way:

Meta makes money from ads. To make more money, they either need to capture more of their users' time and show more ads, or show better ads that users click more often.Meta is betting on AI models making it easier to do both.

Better generative AI means you can make more ads faster, which means there are more ad variants to a/b test across, which means it's easier to find an ad that users will click.

To make users stay on their platforms, Meta figures out what content will keep them there, and then shows them that content. Before gen AI, they were only able to show existing content from real users, but sometimes the "ideal" thing for you hasn't been created yet. They bet on the fact that they'll be able to use AI to create hyper-personalized content for their users that engages them better than human-made content.

Re: Phi 4 available on Ollama

#56

Post by gavmor »

DrPhish wrote: post #37
behnamoh wrote: post #11

[flagged]

It's very hard to put into words without coming off as being unfair to one side or the other, but the ollama project really does provide little-to-no _innovative_ value over simply running components of llama.cpp directly from the command line. 100% of the heavy lifting (from an LLM perspective) is in the llama.cpp codebase. The ollama parts are all simple, well understood, commodity components that most any develope…

Does Ollama offer a GUI? I don't think they do.

I use them because they run as a systemd service with a convenient HTTP API. That's been extremely helpful for switching between GUIs.

I also like their model orgazation scheme, and the modelfile paradigm. It's also really handy that it loads and unloads models when called, which is handy for experimentation and some complex workflows eg embedding followed by inference.

Is llama.cpp doing 100% of the "heavy lifting"? Sure, but some light lifting is also needed to lower the activation threshold and bring the value to life.

I would not use llama.cpp, it's simply too cumbersome.

If Ollama did not exist, I would have to invent it.

Is it not "innovative"? Who cares! I want it. Commodity GUI? Again, I don't think they have a GUI at all. Are you maybe thinking of OpenWebUI?

Re: Phi 4 available on Ollama

#57

Post by miki123211 »

lolinder wrote: post #23
PittleyDunkin wrote: post #17

Earlier quoted context omitted.

> It’s odd that MS is releasing models they are competitors to OA. > I think the strategy is now offer cheap and performant infra to run the models. Is this not what microsoft is doing? What can microsoft possibly lose by releasing a model?

That's exactly what they're saying: it's interesting that Microsoft came to the same conclusion that Meta did, that models are generally not worth keeping locked down. It suggests that OpenAI has a very fragile business model, given that they're wholly dependent on large providers for the infra, which is apparently the valuable part of the equation.

Open AI is the only company that really matters in the consumer conversational AI space.

Their unique value-adds are the Chat GPT brand, being the "default destination" when people want AI, as well as all the "extra features" they add on top of raw LLMs, like the ability to do internet searches, recall facts about you from previous conversations, present data in a nice, interactive way by writing a react app, call down to Python or Wolfram Alpha for arithmetic etc.

I wouldn't be surprised if they eventually stop developing their own models and start using the best ones available at any given time.

Re: Phi 4 available on Ollama

#58

Post by gavmor »

The_Amp_Walrus wrote: post #32
behnamoh wrote: post #11

[flagged]

I might be wrong about this but doesn't ollama do some work to ensure the model runs efficiently given your hardware? Like choosing between how much gpu memory to consume so you don't oom. Does llama.cpp do that for you with zero config?

Yes, Ollama automatically determines the number of layers to offload based on available VRAM.

Re: Phi 4 available on Ollama

#59

Post by sgk284 »

vincent_s wrote: post #47
sgk284 wrote: post #27

Over the holidays, we published a post[1] on using high-precision few-shot examples to get `gpt-4o-mini` to perform similar to `gpt-4o`. I just re-ran that same experiment, but swapped out `gpt-4o-mini` with `phi-4`. `phi-4` really blew me away in terms of learning from few-shots. It measured as being 97% consistent with `gpt-4o` when using high-precision few-shots! Without the few-shots, it was only 37%. That's a hu…

Have you also tried using the large model as FSKD model?

We have, and it works great! We currently do this in production, though we use it to help us optimize for consistency between task executions (vs the linked post, which is about improving the capabilities of a model).

Phrased differently, when a task has many valid and correct conclusions, this technique allows the LLM to see "How did I do similar tasks before?" and it'll tend to solve new tasks by making similar decisions it made for previous similar tasks.

Two things to note:

    - You'll typically still want to have some small epsilon where you choose to run the task without few-shots. This will help prevent mistakes from propagating forward indefinitely.

    - You can have humans correct historical examples, and use their feedback to improve the large model dynamically in real-time. This is basically FSKD where the human is the "large model" and the large foundation model is the "small model".
Post reply on HN