Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI's new open-source model is basically Phi-5

seangoedecke.com

Post reply on HN
Print view View original

161–170 of 233 posts

Re: OpenAI's new open-source model is basically Phi-5

#161

Post by Mkengin »

magicalhippo wrote: post #155
Mkengin wrote: post #141

Earlier quoted context omitted.

Why Qwen2.5 and not Qwen3-30B-A3B-Thinking-2507 or Qwen3-Coder-30B-A3B-Instruct?

Mostly because I had it downloaded already and I'm mostly interested in models that fit on my 16GB GPU. But since you asked, I ran the same questions through both 30B models in the q4_k_m variant, as GPT-OSS 20B is also quantized to about q4. First the ill-posed question: Qwen 3 Coder gave very similar answer to Phi 4, though included a more long-winded explanation in the comments. So not bad, but not great either. Q…

Thank you for testing, I will test GPT-OSS for my use case as well. If you're interested I have 8 GB VRAM, 32 GB RAM and get around 21 token/s with tensor offloading, I would assume that your setup should be even faster than mine with the optimizations. I use the IQ4_KSS quant (by ubergarm on hf) with ik_llama.cpp with this command:

$env:LLAMA_SET_ROWS = "1"; ./llama-server -c 140000 -m D:\ik_llama.cpp\build\bin\Release\models\Qwen3-Coder-30B-A3B-Instruct-IQ4_KSS.gguf -ngl 999 --flash-attn -ctk q8_0 -ctv q8_0 -ot "blk\.(19|2[0-9]|3[0-9]|4[0-7])\.ffn_.*_exps\.=CPU" --temp 0.7 --top-p 0.8 --top-k 20 --repeat_penalty 1.05 --threads 8

In my case I offload layers 19-47, maybe you would just have to offload 37-47, so "blk\.(3[7-9]|4[0-7])\.ffn_.*_exps\.=CPU"

Re: OpenAI's new open-source model is basically Phi-5

#162

Post by subscribed »

fho wrote: post #109
sterlind wrote: post #87

Earlier quoted context omitted.

it's not erotic role-play, but I have a use case of making an AI-powered NetHack clone. specifically, to generate dungeon layouts, dialog for NPCs and to fill in the boatloads of minutae and interactions which NetHack is famous for. you kind of need soul for that, and a lot of background knowledge on mythology/fantasy lore, but also tool use to work the world systems.

Are you getting good results with this? Some time ago I build an "overworld simulator" by having a bunch if jsons that represented villages, characters, building, story hooks and plots. I just asked ChatGPT to "simulate the gameworld by a week". Technically this worked great, but everything was somewhat bland and generic. Noteable not-highlights: - there is a shimmer in the nearby forests that keep villagers up at ni…

I get brilliant, vibrant worlds withots of things happening and the consistent characters with Deepseek V3 (and R1 summaries every so often).

From time to time I get the event so great and character so compelling, I save the best in Author Note or Lorebook.

The overall atmosphere is effortlessly somewhat Skyrim/Game of Thrones/World of Darkness adjacent.

I'd look for these two models to build this simulator of yours, ie R1 to plan and V3 to fill in the blanks. Oh, or maybe Google Gemini 2.5 or 2.0 to plan the story and DeepSeek V3 to fill in.

Re: OpenAI's new open-source model is basically Phi-5

#163

Post by subscribed »

asdffdasy wrote: post #156
wmf wrote: post #20

I saw a bunch of people complaining on Twitter about how GPT-OSS can't be customized or has no soul and I noticed that none of them said what they were trying to accomplish. "The main use-case for fine-tuning small language models is for erotic role-play, and there’s a serious demand." Ah.

why everyone keep pretending very hard that this entire AI summer was not started exclusively by pioneers trying to perfect virtual girlfriend? it's a fact.

Existing models are more than good enough.

Re: OpenAI's new open-source model is basically Phi-5

#164

Post by subscribed »

seydor wrote: post #138
wmf wrote: post #20

I saw a bunch of people complaining on Twitter about how GPT-OSS can't be customized or has no soul and I noticed that none of them said what they were trying to accomplish. "The main use-case for fine-tuning small language models is for erotic role-play, and there’s a serious demand." Ah.

Why would a language company censor language like that? I literally don't see the purpose. Plenty of the best novels have explicit scenes. I certainly don't like novels about infantilized adults. Search engines don't do that, so why do we allow AI companies to do that so blatantly. It's basically culture engineering

Because it's naughty! US is.... prudish, to say the least.

Re: OpenAI's new open-source model is basically Phi-5

#165

Post by egorfine »

> the main use-case for fine-tuning small language models is for erotic role-play

Of course. This is indeed the fact. /s

See that "/s"? I did write it here, but it's suspiciously absent in the original text. Almost makes you think it was typed in all seriousness.

Re: OpenAI's new open-source model is basically Phi-5

#166

Post by katzenversteher »

sterlind wrote: post #87
wmf wrote: post #20

I saw a bunch of people complaining on Twitter about how GPT-OSS can't be customized or has no soul and I noticed that none of them said what they were trying to accomplish. "The main use-case for fine-tuning small language models is for erotic role-play, and there’s a serious demand." Ah.

it's not erotic role-play, but I have a use case of making an AI-powered NetHack clone. specifically, to generate dungeon layouts, dialog for NPCs and to fill in the boatloads of minutae and interactions which NetHack is famous for. you kind of need soul for that, and a lot of background knowledge on mythology/fantasy lore, but also tool use to work the world systems.

I've been experimenting with using various LLMs as a game master for a Lovecraft-inspired role-playing game (not baked into an application, just text-based by prompting). While the LLMs can generate scenarios that fit the theme, they tend to be very generic. I've also noticed that the models are extremely susceptible to suggestion. For example, in one scenario, my investigator was in a bar, and when I commented to another patron, 'Hey, doesn't the barkeeper look a little strange?', the LLM immediately seized on that and turned the barkeeper into an evil, otherworldly creature. This behavior was consistent across all the models I tested. Maybe by prompting the LLM to fully plan the scenario in advance and then adhere to that plan would mitigate the behavior but I haven't tried it. It was just an experiment and I actually had a lot of fun with the behavior. Also the reactions of the LLM if the player does something really unexpected (e.g. "the investigator pulls a sausage out of his pocket and forcefully sticks it into the angry sailors mouth") are sometimes hillarious.

Re: OpenAI's new open-source model is basically Phi-5

#167

Post by eru »

xwolfi wrote: post #129
eru wrote: post #56

Earlier quoted context omitted.

Why? You gotta provide what your customers want. And it's far from sad that we have so many resources, we can give everyone a supercomputer in their pocket just to take selfies and talk about Pokemon. Why would our AIs be any different?

Because we really could use all that time, money and uranium to train them on complex problems we need to solve, rather than entertainment. But you're right, shareholders of OpenAI want profit, not progress, and they'll give us our intellectual Big Mac.

You could say that about any piece of music or movie ever, too. Or any novel.

Re: OpenAI's new open-source model is basically Phi-5

#168

Post by michaelt »

numpad0 wrote: post #128
wmf wrote: post #20

I saw a bunch of people complaining on Twitter about how GPT-OSS can't be customized or has no soul and I noticed that none of them said what they were trying to accomplish. "The main use-case for fine-tuning small language models is for erotic role-play, and there’s a serious demand." Ah.

Maybe I'm just not seeing it, but is that use case really real and not just prudish hallucinations? The market for NSFW novels is smaller than even cyberpunk paperbacks, there's no way everyday people build up addiction for an interactive version of it.

Openrouter provides a multi-model, multi-provider API for hosted LLMs. I myself used their API to compare different LLMs for document classification.

They provide usage rankings [1] and the top 10 applications, in terms of tokens used, are:

1. AI coding agent 2. AI coding agent 3. AI coding agent 4. Library for calling LLMs 5. Role play chat 6. Role play chat 7. Role play chat 8. General purpose chat 9. AI coding agent 10. Role play chat

Certainly, the coding agents are burning through far more tokens. No doubt about that.

And there's undoubtedly a major bias introduced by the fact chatgpt and claude $20/month accounts are heavily discounted, so if your use is SFW why pay more to get an uncensored model?

But overall, to me, the evidence seems pretty robust.

[1] https://openrouter.ai/rankings

Re: OpenAI's new open-source model is basically Phi-5

#169

Post by vintermann »

asdffdasy wrote: post #156
wmf wrote: post #20

I saw a bunch of people complaining on Twitter about how GPT-OSS can't be customized or has no soul and I noticed that none of them said what they were trying to accomplish. "The main use-case for fine-tuning small language models is for erotic role-play, and there’s a serious demand." Ah.

why everyone keep pretending very hard that this entire AI summer was not started exclusively by pioneers trying to perfect virtual girlfriend? it's a fact.

Yes... Specifically by AI dungeon. He got dialog-tuning working (not without some hiccups) before anyone else, and it drew a lot of attention from the API provider.

But then people used it for erotic RP, and it became a PR disaster, and the author blamed his pervert customers. Never mind that certain characters who turned up a lot in AI dungeon stories turned out to be from a fantasy writing site the author had used for fine-tuning material (without permission of course) and he hadn't filtered OUT the dirty stories to put it like that.

Re: OpenAI's new open-source model is basically Phi-5

#170

Post by mrob »

throwuxiytayq wrote: post #132
nickpsecurity wrote: post #114

Earlier quoted context omitted.

[flagged]

i think god did a fairly hack job overall and i’ll gladly en masse commit acts that please me and fail to please his non-existent ass. i’d even turn gay but that’s a bit out of my comfort zone

nickpsecurity's post is currently flagged so I can't reply directly, but I think it's important to clarify that traditional Christian sexual morality aims to maximize population growth. The full list of sexual sins includes one that nickpsecurity missed: sodomy. Sodomy traditionally includes oral sex. The only logical reason to ban oral sex within marriage is to encourage reproduction. Although the Bible does not explicitly forbid birth control, this interpretation is consistent with the demand to "be fruitful and multiply", and forbidding non-procreative sexual activity is mainstream Catholic doctrine.

We've staved off the Malthusian crisis so far, with the Green Revolution and the various social and technological changes that triggered the demographic transition, but evolution does not stop. We already see higher population growth in the highly religious, and religiosity is heritable, especially in adults. This study found 44% heritability for adult religiosity:

https://pubmed.ncbi.nlm.nih.gov/15745438/

It would be a great mistake to assume that overpopulation is no longer a risk, and considering the role that religion plays in it I think traditional Christian sexual morality is a great threat to "stable, prosperous, and peaceful societies" in the long term.

(And even outside marriage oral sex is objectively safer than penetrative sex, although I can understand Christians rejecting the harm-reduction argument.)

Post reply on HN