Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI's new open-source model is basically Phi-5

seangoedecke.com

Post reply on HN
Print view View original

171–180 of 233 posts

Re: OpenAI's new open-source model is basically Phi-5

#171

Post by Havoc »

wkat4242 wrote: post #103
wmf wrote: post #20

I saw a bunch of people complaining on Twitter about how GPT-OSS can't be customized or has no soul and I noticed that none of them said what they were trying to accomplish. "The main use-case for fine-tuning small language models is for erotic role-play, and there’s a serious demand." Ah.

It's not just erotic role play that the censorship affects. My life involves a lot of sexual discussions and that means that everyday talk, chat summaries, email rewrites or translations will cause the model to shut down. I do the latter a lot especially to find colloquialisms because Google translate is often too literal. It's so annoying. Right now I'm using abliterated llama 3.1. I have no need for vision but I wa…

Pretty sure I saw an uncensored version yesterday on localllama

Re: OpenAI's new open-source model is basically Phi-5

#172

Post by magicalhippo »

Mkengin wrote: post #161
magicalhippo wrote: post #155

Earlier quoted context omitted.

Mostly because I had it downloaded already and I'm mostly interested in models that fit on my 16GB GPU. But since you asked, I ran the same questions through both 30B models in the q4_k_m variant, as GPT-OSS 20B is also quantized to about q4. First the ill-posed question: Qwen 3 Coder gave very similar answer to Phi 4, though included a more long-winded explanation in the comments. So not bad, but not great either. Q…

Thank you for testing, I will test GPT-OSS for my use case as well. If you're interested I have 8 GB VRAM, 32 GB RAM and get around 21 token/s with tensor offloading, I would assume that your setup should be even faster than mine with the optimizations. I use the IQ4_KSS quant (by ubergarm on hf) with ik_llama.cpp with this command: $env:LLAMA_SET_ROWS = "1"; ./llama-server -c 140000 -m D:\ik_llama.cpp\build\bin\Rele…

Yeah I think I could get better performance out of both by tweaking, but so far the ease of use has triumphed so far.

Re: OpenAI's new open-source model is basically Phi-5

#173

Post by throwaway98797 »

wmf wrote: post #20

I saw a bunch of people complaining on Twitter about how GPT-OSS can't be customized or has no soul and I noticed that none of them said what they were trying to accomplish. "The main use-case for fine-tuning small language models is for erotic role-play, and there’s a serious demand." Ah.

i tried creating a meme about retards and it refused.

i’m an adult.

Re: OpenAI's new open-source model is basically Phi-5

#174

Post by GTP »

throwaway98797 wrote: post #173
wmf wrote: post #20

I saw a bunch of people complaining on Twitter about how GPT-OSS can't be customized or has no soul and I noticed that none of them said what they were trying to accomplish. "The main use-case for fine-tuning small language models is for erotic role-play, and there’s a serious demand." Ah.

i tried creating a meme about retards and it refused. i’m an adult.

Why is being an adult relevant in any way?

Re: OpenAI's new open-source model is basically Phi-5

#175

Post by Alex-Programs »

wkat4242 wrote: post #103
wmf wrote: post #20

I saw a bunch of people complaining on Twitter about how GPT-OSS can't be customized or has no soul and I noticed that none of them said what they were trying to accomplish. "The main use-case for fine-tuning small language models is for erotic role-play, and there’s a serious demand." Ah.

It's not just erotic role play that the censorship affects. My life involves a lot of sexual discussions and that means that everyday talk, chat summaries, email rewrites or translations will cause the model to shut down. I do the latter a lot especially to find colloquialisms because Google translate is often too literal. It's so annoying. Right now I'm using abliterated llama 3.1. I have no need for vision but I wa…

I'm curious whether you can get https://platform.nuenki.app (my deep translation service; I'm just polishing it before release) to run into "too sexual". It's designed to be resilient to that kind of thing, and it works well in my testing, but maybe you have some better "stress test" content than me!

Re: OpenAI's new open-source model is basically Phi-5

#176

Post by orbital-decay »

I gave it a random sci-fi novel and made it translate a chapter, which is something I do with all models. It refused to discuss minors in sexualized contexts. I was like W.T.F.?! and started bisecting the book, trying to find the piece that triggers this. Turns out there was some absolutely innocent, two sentence long romantic remark involving two secondary 17 years old characters in an unrelated place.

Another issue is that it sometimes has occasional refusals and total meltdowns where it redacts entire paragraphs with placeholder characters, while just trying to casually talk with it about some routine life matters.

That's ridiculous and makes that model garbage at any form of creative writing (including translation) or real life tasks other than math or coding. It has very poor knowledge for a 120B MoE. If you look at the "reasoning" it does, it actually mostly checks the request against the policy.

I thought they must have spent most of their post-training hunting the wrongthink and dumbing the model down as a result, but I can see how the synthetic pretraining data can explain this.

Re: OpenAI's new open-source model is basically Phi-5

#177

Post by xrd »

subscribed wrote: post #162
fho wrote: post #109

Earlier quoted context omitted.

Are you getting good results with this? Some time ago I build an "overworld simulator" by having a bunch if jsons that represented villages, characters, building, story hooks and plots. I just asked ChatGPT to "simulate the gameworld by a week". Technically this worked great, but everything was somewhat bland and generic. Noteable not-highlights: - there is a shimmer in the nearby forests that keep villagers up at ni…

I get brilliant, vibrant worlds withots of things happening and the consistent characters with Deepseek V3 (and R1 summaries every so often). From time to time I get the event so great and character so compelling, I save the best in Author Note or Lorebook. The overall atmosphere is effortlessly somewhat Skyrim/Game of Thrones/World of Darkness adjacent. I'd look for these two models to build this simulator of yours,…

Are you writing or sharing this work anywhere? I would love to read more about your approach.

Re: OpenAI's new open-source model is basically Phi-5

#178

Post by spacecadet »

orbital-decay wrote: post #176

I gave it a random sci-fi novel and made it translate a chapter, which is something I do with all models. It refused to discuss minors in sexualized contexts. I was like W.T.F.?! and started bisecting the book, trying to find the piece that triggers this. Turns out there was some absolutely innocent, two sentence long romantic remark involving two secondary 17 years old characters in an unrelated place. Another issue…

Its a public consumer facing model, not surprised. Go find an unaligned model that will better produce the content you seek...

Re: OpenAI's new open-source model is basically Phi-5

#179

Post by ViktorRay »

orbital-decay wrote: post #176

I gave it a random sci-fi novel and made it translate a chapter, which is something I do with all models. It refused to discuss minors in sexualized contexts. I was like W.T.F.?! and started bisecting the book, trying to find the piece that triggers this. Turns out there was some absolutely innocent, two sentence long romantic remark involving two secondary 17 years old characters in an unrelated place. Another issue…

I wonder what would happen if you asked this model to interact with A Song of Ice and Fire then.

Re: OpenAI's new open-source model is basically Phi-5

#180

Post by vidarh »

cm2012 wrote: post #50
kristopolous wrote: post #48

Earlier quoted context omitted.

This is interesting but there's a little more to it, especially with the erotic. If people were polled what they want to see on social media, few would say things that are inflammatory, upsetting, divisive, etc but those as we know are strong drivers of engagement. It's because you're polling for affinity or disclosed preference not for the actual engagement drivers. For instance, if a male says they watch male porno…

Man, I would pay money to see the (anonymized) trends on an adult website. Fascinating view into such an under studied area of humanity nature. I bet the porn tubes have data that sociologists could write papers on.

Pornhub does yearly roundups of stats, as well as for various events:

https://www.pornhub.com/insights/2024-year-in-review

https://www.pornhub.com/insights/

Post reply on HN