Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Phi 4 available on Ollama

ollama.com

Post reply on HN
Print view View original

91–100 of 138 posts

Re: Phi 4 available on Ollama

#91

Post by api »

OhioMan2943 wrote: post #67

Is anyone blown away by how fast we got to running something this powerful locally? I know it's easy to get burnt out on llms but this is pretty incredible. I genuinely think we're only 2 years away from full custom local voice to voice llm assistants that grow with you like JOI in BR2049 and it's going to change how we think about being human and being social, and how we grow up.

I’ve thought for a while that Joi in BR2049 was less dystopian than what we will probably do with AI. She doesn’t constantly prompt K to buy more credits (like a mobile game) to continue engaging with her or deepen their relationship. (“If you really love me…”) I’ve been expecting that this is how our industry would operate given the customer hostile psychologically abusive hellscape of social and mobile. Of course there’s still time.

She appears to be a local model runnable on a small device without cloud.

Re: Phi 4 available on Ollama

#92

Post by mistercheph »

buyucu wrote: post #77

I have unfortunately been disappointed with the llama.cpp/ollama ecosystem of late, and thinking about moving my things to vllm instead. llama.cpp basically dropped support for multimodal visual models. ollama still does support them, but only a handful. Also ollama still does not support vulkan eventhough llama.cpp had vulkan support for a long long time now. This has been very sad to watch. I'm more and more convin…

Ollama maintainers seem a bit burned out

Re: Phi 4 available on Ollama

#93

Post by driverdan »

mythz wrote: post #6

Was disappointed in all the Phi models before this, whose benchmark results scored way better than it worked in practice, but I've been really impressed with how good Phi-4 is at just 14B. We've run it against the top 1000 most popular StackOverflow questions and it came up 3rd beating out GPT-4 and Sonnet 3.5 in our benchmarks, only behind DeepSeek v3 and WizardLM 8x22B [1]. We're using Mixtral 8x7B to grade the qua…

IMO SO questions is not a good evaluation. These models were likely trained on the top 1000 most popular StackOverflow questions. You'd expect them to have similar results and perform well when compared to the original answers.

Re: Phi 4 available on Ollama

#94

Post by brookst »

api wrote: post #91
OhioMan2943 wrote: post #67

Is anyone blown away by how fast we got to running something this powerful locally? I know it's easy to get burnt out on llms but this is pretty incredible. I genuinely think we're only 2 years away from full custom local voice to voice llm assistants that grow with you like JOI in BR2049 and it's going to change how we think about being human and being social, and how we grow up.

I’ve thought for a while that Joi in BR2049 was less dystopian than what we will probably do with AI. She doesn’t constantly prompt K to buy more credits (like a mobile game) to continue engaging with her or deepen their relationship. (“If you really love me…”) I’ve been expecting that this is how our industry would operate given the customer hostile psychologically abusive hellscape of social and mobile. Of course t…

I expect AI to be like any other tech: some fantastic uses that advance humanity and improve the world, some terrible uses that abuse, manipulate, oppress.

I don’t see anything in the tech that indicates a singular pattern that will be “good” or “bad”.

Re: Phi 4 available on Ollama

#95

Post by lmeyerov »

sgk284 wrote: post #61
lmeyerov wrote: post #53

Earlier quoted context omitted.

I like the direction, but have a pretty different experience in practice. This spans legal analytics, social media analytics, code synthesis, news analysis, cyber security LLMs, etc: 1. The only ultimate absolute quality metric I saw in that blogpost afaict was expert agreement... at 90%. All of our customers would fire us at that level across all of the diff b2b domains we work in. I'm surprised 90% is considered ac…

re: 90% – this particular case is a fairly subjective and creative task, where humans (and the LLM) are asked to follow a 22 page SOP. They've had a team of humans doing the task for 9 years, with exceptionally high variance in performance. The blended performance of the human team is meaningfully below this 90% threshold (~76%) – which speaks to the difficulty of the task. It's, admittedly, a tough task to measure o…

I'm more confused now. If this is a tough and high-value task, we would not use gpt-4o-mini on its own, eg, add more steps like a verifier & retry, or just do gpt-4o to begin with, and would more seriously consider fine-tuning in addition to the prompt engineering. The blog argued against that, but maybe I read too quickly.

And agreed, people expect $ they invest into computer systems to do much better than their bad & avg employees. AI systems get the added challenge where they must do ~100% on what non-AI rules would catch ("why are you using AI?") + extra lift from AI ("what did this add?"). We generally get evaluated on matching experts (low bar), and exceeding them (high bar). Comparing to average staff is, frustratingly, a breakout.

Each scenario is different obviously..

Re: Phi 4 available on Ollama

#96

Post by brookst »

PittleyDunkin wrote: post #86
krick wrote: post #29

Earlier quoted context omitted.

To be fair, OpenAI's products are not really models, they are... products. So it's debatable if they really do have anything special. I don't really think they do, because to me it seemed pretty much since GPT-1, that having callbacks to run python and query google, having "inner dialog" before summarizing an answer and a dozen more simple improvements like this are quite obvious things to do, that nobody just actual…

> To be fair, OpenAI's products are not really models, they are... products What's the distinction? What kind of functionality do they offer that other models don't?

A model is an ingredient in an AI product. The product includes the UI, tools / RAG, apps on various platforms, system prompts and personality, and so on.

Lots of products have been successful without a technical moat. Facebook has network effects, Apple has UX (though silicon has become a technical advantage if not moat), Adobe has “everyone knows how to use these tools” switching costs, Google has brand synonymous with search.

Companies are betting that models will be commodities but AI products will be sticky.

Re: Phi 4 available on Ollama

#97

Post by cloudking »

OhioMan2943 wrote: post #67

Is anyone blown away by how fast we got to running something this powerful locally? I know it's easy to get burnt out on llms but this is pretty incredible. I genuinely think we're only 2 years away from full custom local voice to voice llm assistants that grow with you like JOI in BR2049 and it's going to change how we think about being human and being social, and how we grow up.

Not related to local LLMs, but JOI from BR2049 is essentially what Replika is striving for: https://replika.com/

Infact during the onboarding process they ask the user to choose which AI companion movie they related to the most: Her, BR2049 or Ex-Machina. The experience is then tailored to align closer to the movie chosen.

It's quite a terrible app from a product design perspective: filled with dark patterns (like sending the user blurred images to "unlock*) and upsells, but it's become successful amongst the masses that have adopted it, which I find fascinating. 30m+ users https://en.wikipedia.org/wiki/Replika#:~:text=Replika%20beca....

Re: Phi 4 available on Ollama

#98

Post by ericrosedev »

brookst wrote: post #94
api wrote: post #91

Earlier quoted context omitted.

I’ve thought for a while that Joi in BR2049 was less dystopian than what we will probably do with AI. She doesn’t constantly prompt K to buy more credits (like a mobile game) to continue engaging with her or deepen their relationship. (“If you really love me…”) I’ve been expecting that this is how our industry would operate given the customer hostile psychologically abusive hellscape of social and mobile. Of course t…

I expect AI to be like any other tech: some fantastic uses that advance humanity and improve the world, some terrible uses that abuse, manipulate, oppress. I don’t see anything in the tech that indicates a singular pattern that will be “good” or “bad”.

The enshitification trend seems pretty dominant, and pretty bad for users / good for investors

Re: Phi 4 available on Ollama

#99

Post by zitterbewegung »

lmeyerov wrote: post #95
sgk284 wrote: post #61

Earlier quoted context omitted.

re: 90% – this particular case is a fairly subjective and creative task, where humans (and the LLM) are asked to follow a 22 page SOP. They've had a team of humans doing the task for 9 years, with exceptionally high variance in performance. The blended performance of the human team is meaningfully below this 90% threshold (~76%) – which speaks to the difficulty of the task. It's, admittedly, a tough task to measure o…

I'm more confused now. If this is a tough and high-value task, we would not use gpt-4o-mini on its own, eg, add more steps like a verifier & retry, or just do gpt-4o to begin with, and would more seriously consider fine-tuning in addition to the prompt engineering. The blog argued against that, but maybe I read too quickly. And agreed, people expect $ they invest into computer systems to do much better than their bad…

I feel like LLMs are going to be a skill to have similar to the ability to google or type since it can get good answers pretty well but bad answers when you don't know the subject manner.

Re: Phi 4 available on Ollama

#100

Post by redcobra762 »

k__ wrote: post #81

"built upon a blend of synthetic datasets, data from filtered public domain websites, and acquired academic books and Q&A datasets" Does this mean the model was trained without copyright infringements?

This is a presumptive question, as training AI models may fall under fair use.
Post reply on HN