Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Phi 4 available on Ollama

ollama.com

Post reply on HN
Print view View original

121–130 of 138 posts

Re: Phi 4 available on Ollama

#121

Post by throw234234234 »

imiric wrote: post #109
simonw wrote: post #88

Earlier quoted context omitted.

I should emphasize that I really don't think the dystopian version of this is likely to happen - the one where "AGI/ASI" puts every human out of work and society collapses. Human beings have agency, and we are very good at rolling with the punches. We've survived waves of automation for hundreds of years. I'm much more confident that we will continue to find ways to use these things as tools that elevate us, not repl…

> We've survived waves of automation for hundreds of years. I'm much more confident that we will continue to find ways to use these things as tools that elevate us, not replace us. The difference with past technological breakthroughs is that they augmented what humans could do, but didn't have the potential to replace human labor altogether as AI does. They were disruptive, but humans were able to adapt to new career…

As many forums say, with other tech inventions they replaced the horse not the rider. With AI; they are replacing the rider - that makes it a unique technology that does not compare to previous technology being introduced. Other forms of technology typically enabled use cases which didn't seem possible (e.g. electricity, cooking food faster, flying, etc) - this one at present is just about making existing cases more efficient/removing the need for labor. As many non-techies mention - other than doing my assignment/email/etc what benefit does it have on my daily life other than threaten some jobs and generate some worthless online content?

The cost/benefit for the labor/middle/low classes is at best low right now. I define that as someone who needs to trade time to continue surviving as an ongoing concern even if they have some wealth behind them.

I think the outcome where any form of meritocratic society gives way to old fashioned resource acquisition based societies is definitely one believable outcome. Warfare, land and resource ownership - the old will become the new again.

Re: Phi 4 available on Ollama

#123

Post by kuatroka »

OJFord wrote: post #118
kuatroka wrote: post #110

I’ve pulled and ran it. It launches fine, but when I actually ask it anything I constantly get just a blank line. Does anyone else experience this?

I would guess on your hardware you're getting <1 token/time-you've-bothered-waiting?

not sure what it means. I've got macbook pro M1 Max with 64Gb. Any other model runs perfectly fine. Only Phi4 blanks on me

Re: Phi 4 available on Ollama

#124

Post by potatoman22 »

lmeyerov wrote: post #117
potatoman22 wrote: post #112

Earlier quoted context omitted.

What's your loop for prompt engineering with GPT-4o? Do you feed the meta-prompter the misclassified examples? Also does the evaluation drive the synthetic data production almost like boosting?

'it varies' b/c we do everything from an interactive analytics chat agent (loiue.ai UI) to data-intensive continuous-monitoring (louie.ai pipelines) to one-off customer assists like $B court cases 1. Common themes in our development-time loop: * We don't do synthetic data. We do real data or anonymized data. When we lack data, we go and get some. That may mean paying people, doing it ourselves, setting up simulation…

Thanks! I love your focus on evaluation, it's missing in a lot of LLM products. I worked in the medical field and we valued model validation with similar importance. Our processes sound similar, too. One difference is that our customers still saw utility in models with much lower F1 than 90%. Rare events are hard to predict.

Re: Phi 4 available on Ollama

#125

Post by OhioMan2943 »

actsasbuffoon wrote: post #108
simonw wrote: post #88

Earlier quoted context omitted.

I should emphasize that I really don't think the dystopian version of this is likely to happen - the one where "AGI/ASI" puts every human out of work and society collapses. Human beings have agency, and we are very good at rolling with the punches. We've survived waves of automation for hundreds of years. I'm much more confident that we will continue to find ways to use these things as tools that elevate us, not repl…

I’m on the opposite end of the spectrum. I’m almost certain that this is going to end extremely badly for the majority of humanity, and for programmers in particular. I think there’s a less than 5% chance that this goes well, and that’s only if we get a series of things to go extremely well. And frankly, we’re tracking along the extremely bad path so far.

We barely survived one nuclear arms race, and this could give every nation state a new type of power weapon every 5ish years through the inevitable scaling in energy and weapons. I agree we're on one of the worst timelines for AI/AGI/ASI with the world actively being run into the ground by short sighted 'dementia-ocracies' and every security risk about to increase dramatically.

Re: Phi 4 available on Ollama

#126

Post by OhioMan2943 »

simonw wrote: post #88
OhioMan2943 wrote: post #73

Earlier quoted context omitted.

I'm in complete agreement with your more recent timeline piece (the negative one), and as a younger user (22 year old student) I'm actively relocating this year to somewhere slightly more rural with a focus on physical/knowledge combined work to secure a good quality of life nearly solely because of how fast our timelines are. A 'word calculator' this effective is the best substitute that we have for a logic calculat…

I should emphasize that I really don't think the dystopian version of this is likely to happen - the one where "AGI/ASI" puts every human out of work and society collapses. Human beings have agency, and we are very good at rolling with the punches. We've survived waves of automation for hundreds of years. I'm much more confident that we will continue to find ways to use these things as tools that elevate us, not repl…

I agree people are way more agentic than we give them credit for in these situations. We tend to 'petri-dish' ourselves and act like we're just the products of our environments, being swept along when top-down analysing large situations like this when that really isn't the case.

That being said, I can't see any world where there isn't mass ontological shock/hysteria, mass unemployment, and unrest at least for a few years, and I feel like it is definitely the kind of event you take active measures and preparations for beforehand.

And so do I, but like the golden rule of camping you should prepare for the worst and hope for the best!

Re: Phi 4 available on Ollama

#127

Post by sgk284 »

lmeyerov wrote: post #95
sgk284 wrote: post #61

Earlier quoted context omitted.

re: 90% – this particular case is a fairly subjective and creative task, where humans (and the LLM) are asked to follow a 22 page SOP. They've had a team of humans doing the task for 9 years, with exceptionally high variance in performance. The blended performance of the human team is meaningfully below this 90% threshold (~76%) – which speaks to the difficulty of the task. It's, admittedly, a tough task to measure o…

I'm more confused now. If this is a tough and high-value task, we would not use gpt-4o-mini on its own, eg, add more steps like a verifier & retry, or just do gpt-4o to begin with, and would more seriously consider fine-tuning in addition to the prompt engineering. The blog argued against that, but maybe I read too quickly. And agreed, people expect $ they invest into computer systems to do much better than their bad…

One point of confusion might be that this is a tough but relatively low-value task (on a per-unit basis). The budget per item moderated is measured in small double-digit cents, but there's hundreds of thousands of items regularly being ingested.

FWIW – across all of these, we already do automated prompt rewriting, self-reflection, verification, and a suite of other things that help maximize reliability / quality, but those tokens add up quickly and being able to dynamically switch over to a smaller model without degrading performance improves margin substantially.

Fine-tuning is a non-starter for a number of reasons, but that's a much longer post.

Re: Phi 4 available on Ollama

#128

Post by ls612 »

mlepath wrote: post #43
crorella wrote: post #9

It’s odd that MS is releasing models they are competitors to OA. This reinforce the idea that there is no real strategic advantage in owning a model. I think the strategy is now offer cheap and performant infra to run the models.

> This reinforce the idea that there is no real strategic advantage in owning a model For these models probably no. But for proprietary things that are mission critical and purpose-built (think Adobe Creative Suite) the calculus is very different. MS, Google, Amazon all win from infra for open source models. I have no idea what game Meta is playing

Meta seems to be playing the “commoditize your complements” game. Which is good for the rest of us who get close to SotA open weights models.

Re: Phi 4 available on Ollama

#129

Post by blharr »

mettamage wrote: post #122

How come models can be so small now? I don't know a lot about AI, but is there an ELI5 for a software engineer that knows a bit about AI? For context: I've made some simple neural nets with backprop. I read [1]. [1] http://neuralnetworksanddeeplearning.com/

You can find the phi-4 technical report [here](https://www.microsoft.com/en-us/research/uploads/prod/2024/1...)

The brief of it is by curating a smaller synthetic dataset of high quality from textbooks, problem sets, etc. instead of dumping a massive dataset with tons of information.

Re: Phi 4 available on Ollama

#130

Post by k__ »

redcobra762 wrote: post #100
k__ wrote: post #81

"built upon a blend of synthetic datasets, data from filtered public domain websites, and acquired academic books and Q&A datasets" Does this mean the model was trained without copyright infringements?

This is a presumptive question, as training AI models may fall under fair use.

Just because some laws define fair use in some kind of way, it doesn't mean potential customers see it that way.
Post reply on HN