Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Phi 4 available on Ollama

ollama.com

Post reply on HN
Print view View original

31–40 of 138 posts

Re: Phi 4 available on Ollama

#31

Post by m3kw9 »

sumedh wrote: post #26
lolinder wrote: post #23

Earlier quoted context omitted.

That's exactly what they're saying: it's interesting that Microsoft came to the same conclusion that Meta did, that models are generally not worth keeping locked down. It suggests that OpenAI has a very fragile business model, given that they're wholly dependent on large providers for the infra, which is apparently the valuable part of the equation.

> It suggests that OpenAI has a very fragile business model That is the reason they are making products so that people stay on the platform.

OpenAI has infrastructure and a product around serving to people plus they have SOTA models. Joe blow can’t just take a Qwen or whatever and start making money at scale

Re: Phi 4 available on Ollama

#33

Post by wongarsu »

lolinder wrote: post #28
sumedh wrote: post #26

Earlier quoted context omitted.

> It suggests that OpenAI has a very fragile business model That is the reason they are making products so that people stay on the platform.

Their big risk there as I see it is that the market for "I need an AI" is much much smaller than they thought it would be. People don't generally need or want to pay for "AI", they want to pay for solutions to specific problems. This means that in a world where AWS/Azure/GCP all compete in the compute and the models themselves are commodities, AI isn't a product, it's a feature of every product. In that world, what i…

The ones at the forefront of the "I need an AI" hype are selling agents, or tools that integrate in your email workflow, or some other tool with AI in the name. OpenAI is selling the shovels, the backend API those services are using. AWS/Azure/GCP are selling factory space and are providing blue-prints for shovels. Which is compelling at scale, but if you are busy selling AI tools to people who don't know better it's faster to just use an API to whatever OpenAI offering is SOTA or close to SOTA.

I'd agree there isn't much money in it. OpenAI should probably milk the revenue they get now and make hay while the sun is shining. But their apparent strategy is to bet it all on finding another breakthrough similar to the switch from text completion to a chat interface

Re: Phi 4 available on Ollama

#34

Post by lolinder »

mythz wrote: post #14
KTibow wrote: post #8

Earlier quoted context omitted.

Interesting eval but my first reaction is "using Mixtral as a judge doesn't sound like a good idea". Have you tested how different its results are from GPT-4 as a judge (on a small scale) or how stuff like style and order can affect its judgements? Edit: they have a blog post https://pvq.app/posts/individual-voting-comparison although it could go deeper

Yeah we evaluated several models for grading ~1 year ago and concluded Mixtral was the best choice for us, as it was the best model yielding the best results that we could self-host and distribute the load of grading 1.2M+ answers over several GPU Servers. We would have liked to pick a neutral model like Gemini which was fast, reliable and low cost, unfortunately it gave too many poor answers good grades [1]. If we h…

Honestly, the fact that you used an LLM to grade the answers at all is enough to make me discount your results entirely. That it showed obvious preference to the model with which it shares weights is just a symptom of the core problem, which is that you had to pick a model to trust before you even ran the benchmarks.

The only judges that matter at this stage are humans. Maybe someday when we have models that humans agree are reliably good you could use them to judge lesser-but-cheaper models.

Re: Phi 4 available on Ollama

#35

Post by lolinder »

wongarsu wrote: post #33
lolinder wrote: post #28

Earlier quoted context omitted.

Their big risk there as I see it is that the market for "I need an AI" is much much smaller than they thought it would be. People don't generally need or want to pay for "AI", they want to pay for solutions to specific problems. This means that in a world where AWS/Azure/GCP all compete in the compute and the models themselves are commodities, AI isn't a product, it's a feature of every product. In that world, what i…

The ones at the forefront of the "I need an AI" hype are selling agents, or tools that integrate in your email workflow, or some other tool with AI in the name. OpenAI is selling the shovels, the backend API those services are using. AWS/Azure/GCP are selling factory space and are providing blue-prints for shovels. Which is compelling at scale, but if you are busy selling AI tools to people who don't know better it's…

Yeah, the problem with selling shovels where shovels=APIs is that APIs cost almost nothing to replicate and are not copyrightable. Tools like Ollama and LiteLLM already offer APIs that are drop-in replacements for OpenAI.

OpenAI isn't losing yet because their models are still marginally better and they have a lot of inertia, but their API isn't going to save them.

> But their apparent strategy is to bet it all on finding another breakthrough similar to the switch from text completion to a chat interface

I'm still convinced that their strategy is to find an exit ASAP and let Altman cash out. He's playing up AGI because it's the only possible way that "AI" becomes a product in its own right so investors need to hear that that's the goal, but I think he knows full well it's not in reach and he can only keep the con going so long. An exit is the most profitable way out for him.

Re: Phi 4 available on Ollama

#36

Post by bongodongobob »

krick wrote: post #29
lolinder wrote: post #23

Earlier quoted context omitted.

That's exactly what they're saying: it's interesting that Microsoft came to the same conclusion that Meta did, that models are generally not worth keeping locked down. It suggests that OpenAI has a very fragile business model, given that they're wholly dependent on large providers for the infra, which is apparently the valuable part of the equation.

To be fair, OpenAI's products are not really models, they are... products. So it's debatable if they really do have anything special. I don't really think they do, because to me it seemed pretty much since GPT-1, that having callbacks to run python and query google, having "inner dialog" before summarizing an answer and a dozen more simple improvements like this are quite obvious things to do, that nobody just actual…

What product? A chat window? I'm not trying to be rude btw, but if the product isn't the LLM itself, that's all they have.

Re: Phi 4 available on Ollama

#37

Post by DrPhish »

behnamoh wrote: post #11

[flagged]

It's very hard to put into words without coming off as being unfair to one side or the other, but the ollama project really does provide little-to-no _innovative_ value over simply running components of llama.cpp directly from the command line. 100% of the heavy lifting (from an LLM perspective) is in the llama.cpp codebase. The ollama parts are all simple, well understood, commodity components that most any developer could have produced.

Now, applications like ollama obviously need to exist, as not everyone can run CLI utilities, let alone clone a git repo and compile themselves. Easy to use GUIs are essential for the adoption of new tech (much like how there are many apps that wrap ffmpeg and are mostly UI).

However, if ollama are mostly doing commodity GUI things over a fully fleshed-out, _unique_ codebase to which their very existence is owed, they should do everything in their power to point that out. I'm sure they're legally within their rights because of the licensing, but just from an ethical perspective.

I think there is a lot of ill-will towards ollama in some hard-core OG LLM communities because ollama appears to be attempting to capture the value that ggerganov has provided to the world in this tool without adequate attribution (although there is a small footnote, iirc). Basically, the debt that ollama owes to llama.cpp is so immense that they need to do a much better job recognizing it imo.

Re: Phi 4 available on Ollama

#38

Post by mythz »

lolinder wrote: post #34
mythz wrote: post #14

Earlier quoted context omitted.

Yeah we evaluated several models for grading ~1 year ago and concluded Mixtral was the best choice for us, as it was the best model yielding the best results that we could self-host and distribute the load of grading 1.2M+ answers over several GPU Servers. We would have liked to pick a neutral model like Gemini which was fast, reliable and low cost, unfortunately it gave too many poor answers good grades [1]. If we h…

Honestly, the fact that you used an LLM to grade the answers at all is enough to make me discount your results entirely. That it showed obvious preference to the model with which it shares weights is just a symptom of the core problem, which is that you had to pick a model to trust before you even ran the benchmarks. The only judges that matter at this stage are humans. Maybe someday when we have models that humans a…

The Mixtral grading model calculates the original starting votes which can be further influenced by Users voting on their preferred answer which affects the leaderboard standings.

It should be noted that Mixtral 8x7B didn't grade its own model very high at 11th, it's standout was grading Microsoft's WizardLM2 model pretty high at #2. Although it's not entirely without merit as at the time of release it was Microsoft's most advanced model and the best opensource LLM available [1]. Which we also found generated great high quality answers which I'm surprised it's not more used as it's only OpenRouter's 15th most used model this month [2], although it's received very little marketing behind it, essentially just an announcement blog post.

Whilst nothing is perfect we're happy with the Grading system as it's still able to identify good answers from bad ones, good models from bad ones and which topics models perform poorly on. Some of the grades are surprising since we have prejudices on where models should rank before the results are concluded, which is also why it's important to have multiple independent benchmarks, especially benchmarks that LLMs aren't optimized for as I've often been disappointed by how some models perform in practice vs how well they perform in benchmarks.

Either way you can inspect the different answers from the different models yourself by paging through the popular questions [3]:

[1] https://wizardlm.github.io/WizardLM2/

[2] https://openrouter.ai/rankings?view=month

[3] https://pvq.app/questions

Re: Phi 4 available on Ollama

#39

Post by ta988 »

lolinder wrote: post #28
sumedh wrote: post #26

Earlier quoted context omitted.

> It suggests that OpenAI has a very fragile business model That is the reason they are making products so that people stay on the platform.

Their big risk there as I see it is that the market for "I need an AI" is much much smaller than they thought it would be. People don't generally need or want to pay for "AI", they want to pay for solutions to specific problems. This means that in a world where AWS/Azure/GCP all compete in the compute and the models themselves are commodities, AI isn't a product, it's a feature of every product. In that world, what i…

They are the useful idiots that attracted the funding to take the risks and make the technology emerge but didn't have the right marketing and political power. They will disappear as fast as they appeared. It is a common tale in technologies, a lot of companies who invented and/or developed something and did all the hard work just couldn't compete when it got comoditized.

Re: Phi 4 available on Ollama

#40

Post by blackqueeriroh »

bongodongobob wrote: post #36
krick wrote: post #29

Earlier quoted context omitted.

To be fair, OpenAI's products are not really models, they are... products. So it's debatable if they really do have anything special. I don't really think they do, because to me it seemed pretty much since GPT-1, that having callbacks to run python and query google, having "inner dialog" before summarizing an answer and a dozen more simple improvements like this are quite obvious things to do, that nobody just actual…

What product? A chat window? I'm not trying to be rude btw, but if the product isn't the LLM itself, that's all they have.

Interesting, so apparently UI and UX and responsiveness and polish all don’t matter for products? We can just ship shittily drawn interfaces now?
Post reply on HN