Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI's new open-source model is basically Phi-5

seangoedecke.com

Post reply on HN
Print view View original

221–230 of 233 posts

Re: OpenAI's new open-source model is basically Phi-5

#221

Post by sekh60 »

ViktorRay wrote: post #179
orbital-decay wrote: post #176

I gave it a random sci-fi novel and made it translate a chapter, which is something I do with all models. It refused to discuss minors in sexualized contexts. I was like W.T.F.?! and started bisecting the book, trying to find the piece that triggers this. Turns out there was some absolutely innocent, two sentence long romantic remark involving two secondary 17 years old characters in an unrelated place. Another issue…

I wonder what would happen if you asked this model to interact with A Song of Ice and Fire then.

Depends, is Mastercard OpenAI's payment processor?

Re: OpenAI's new open-source model is basically Phi-5

#222

Post by wkat4242 »

CjHuber wrote: post #220
wkat4242 wrote: post #184

Earlier quoted context omitted.

Hmm? For me it's still the best model for general use. That I've tried at least. Gemma and phi I didn't like so much. Qwen is chinese and I've had bad experiences with those (reverting to Chinese when they get confused). Not for knowledge, but I combine it with searches of course. None of the small models are good at knowledge.

So you are using local models for general use? For me when it’s not ultra sensitive information, I don‘t want a just good enough LLM so I use the API of a proper large one.

For me good enough is usually good enough (and pretty excellent). Unless I ask for something really complex, then I use perplexity in research mode.

But I have everything in OpenWebUI so I can choose with a touch of a button. Sometimes I wonder whether GPT could have done better and I try it and it's usually not significantly better than llama 3.1 8b

Re: OpenAI's new open-source model is basically Phi-5

#223

Post by wkat4242 »

CjHuber wrote: post #220
wkat4242 wrote: post #184

Earlier quoted context omitted.

Hmm? For me it's still the best model for general use. That I've tried at least. Gemma and phi I didn't like so much. Qwen is chinese and I've had bad experiences with those (reverting to Chinese when they get confused). Not for knowledge, but I combine it with searches of course. None of the small models are good at knowledge.

So you are using local models for general use? For me when it’s not ultra sensitive information, I don‘t want a just good enough LLM so I use the API of a proper large one.

By the way, I usually use the AI for filtering, summarisation etc. Not for facts (like "what would be the best model of CPU I could buy" or "what is the capital of Estonia". When I do that I will link it to web searches anyway. Small models are indeed pretty useless for asking facts from. But big models can't be relied on either. They still hallucinate a lot. So I tend to combine it with web searches so I also have the references. But in general the whole fact thing for me is not a big usecase for AI.

Re: OpenAI's new open-source model is basically Phi-5

#224

Post by bobsmooth »

krick wrote: post #212
bobsmooth wrote: post #198

Earlier quoted context omitted.

After running Dolphin Mistral on my own machine I won't trust censored models ever again.

Recently I somehow wasn't using LLMs locally and relied mostly on ChatGPT for casual tasks. I think it was a little less than a year since I played with ollama, and I remember that my impression was that all recent popular models definitely aren't "uncensored" in a sense that some older modification of llama2 I used was, and all suck for prose-related tasks anyway. In fact, nothing but ChatGPT models seemed good enou…

The one I really like is Dolphin-Mistral-24B-Venice-Edition-GGUF. And yeah, it's really uncensored.

Re: OpenAI's new open-source model is basically Phi-5

#225

Post by CjHuber »

wkat4242 wrote: post #223
CjHuber wrote: post #220

Earlier quoted context omitted.

So you are using local models for general use? For me when it’s not ultra sensitive information, I don‘t want a just good enough LLM so I use the API of a proper large one.

By the way, I usually use the AI for filtering, summarisation etc. Not for facts (like "what would be the best model of CPU I could buy" or "what is the capital of Estonia". When I do that I will link it to web searches anyway. Small models are indeed pretty useless for asking facts from. But big models can't be relied on either. They still hallucinate a lot. So I tend to combine it with web searches so I also have t…

I understand your take, however my main use case is for the LLM to find subtle connections and biases in text or analyse nuances, so I figure more parms do make adifference in my case. Also in my experience it does also make a difference in summarization and filtering tasks, as a larger param one possibly knows better what is important and relevant in a text

Re: OpenAI's new open-source model is basically Phi-5

#226

Post by wkat4242 »

CjHuber wrote: post #225
wkat4242 wrote: post #223

Earlier quoted context omitted.

By the way, I usually use the AI for filtering, summarisation etc. Not for facts (like "what would be the best model of CPU I could buy" or "what is the capital of Estonia". When I do that I will link it to web searches anyway. Small models are indeed pretty useless for asking facts from. But big models can't be relied on either. They still hallucinate a lot. So I tend to combine it with web searches so I also have t…

I understand your take, however my main use case is for the LLM to find subtle connections and biases in text or analyse nuances, so I figure more parms do make adifference in my case. Also in my experience it does also make a difference in summarization and filtering tasks, as a larger param one possibly knows better what is important and relevant in a text

I can recommend to try a comparison if you can (like with OpenWebUI, you can simply make it run the same prompts through multiple models). If you have access to local models of course. Though you could also use some models like llama on groq.

My usecases are different from yours of course but I don't really see significant difference in result quality in most cases.

Re: OpenAI's new open-source model is basically Phi-5

#228

Post by katzenversteher »

Gracana wrote: post #191
katzenversteher wrote: post #166

Earlier quoted context omitted.

I've been experimenting with using various LLMs as a game master for a Lovecraft-inspired role-playing game (not baked into an application, just text-based by prompting). While the LLMs can generate scenarios that fit the theme, they tend to be very generic. I've also noticed that the models are extremely susceptible to suggestion. For example, in one scenario, my investigator was in a bar, and when I commented to an…

Have you used any thinking models? I remember being surprised by QwQ-32B when I tried it. It would think about what I said and how it should respond, reiterate the behaviors I had assigned to it, and respond accordingly. That constant self-reinforcement in the thinking phase seemed to keep it on track.

I might have tried DeepSeek-r1-8B in the past but I'll certainly try again. Good idea.

Re: OpenAI's new open-source model is basically Phi-5

#229

Post by katzenversteher »

GrinningFool wrote: post #203
katzenversteher wrote: post #166

Earlier quoted context omitted.

I've been experimenting with using various LLMs as a game master for a Lovecraft-inspired role-playing game (not baked into an application, just text-based by prompting). While the LLMs can generate scenarios that fit the theme, they tend to be very generic. I've also noticed that the models are extremely susceptible to suggestion. For example, in one scenario, my investigator was in a bar, and when I commented to an…

> on, 'Hey, doesn't the barkeeper look a little strange?', the LLM immediately seized on that and turned the barkeeper into an evil, otherworldly creature. Though making it an evil otherwordly creature is a bit extreme, it's at least similar to what a flexible GM can do. In my DMing days, I would often develop new paths that integrated into the whole inspired by things my players noticed/suspected.

In my GM days, I had a lot of trouble with players that tried really their best to completely leave the path I prepared for them.

You are right though and it's not that I completely dislike the LLMs "flexibilty" and openness to suggestions. However, it's also super easy to use it for "cheating". E.g. it generated a scenario with an evil entity about to attack me and some friendly NPC and I could "solve" that problem by telling the NPC "remember the device I gave you last week and told you to always keep on hand? pull the trigger now!" (that never happend, at least to the LLMs knowledge) and the LLM made up some device that shot a beam of magic light at the creature and stopped it.

Re: OpenAI's new open-source model is basically Phi-5

#230

Post by naasking »

fc417fc802 wrote: post #219
naasking wrote: post #215

Earlier quoted context omitted.

There are all sorts of adjacent reasons too, like human trafficking and prostitution. It's probably similar reasons to why landlords are so aggressive about renters not doing any sex work, because even indirectly receiving proceeds from prostitution is illegal, even when you're unaware of it.

In which jurisdictions? I'm skeptical. If a vendor takes reasonable precautions it should not generally be possible to hold them liable. Anyway you've circled back to user generated content. But again, that's far from the only thing that payment processors have discriminated against over the past couple of decades.

This is a big feature of the Nordic model, but it's not uncommon in other legal frameworks either.
Post reply on HN