Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI's new open-source model is basically Phi-5

seangoedecke.com

Post reply on HN
Print view View original

21–30 of 233 posts

Re: OpenAI's new open-source model is basically Phi-5

#21

Post by charcircuit »

jononor wrote: post #3
NitpickLawyer wrote: post #2

Yeah, makes sense. Good observations regarding the benchmark vs. vibes in general, and I didn't know / made the connection between the lead of phi models going to oAI and gpt-oss. Could very well be a similar exercise + their "new" prompt level adherence (system > developer > user). In all the traces I've seen of refusals the model "quotes" the policy quite religiously. Similar thing was announced for gpt5. I think t…

No the preferred way of making modifications is the weights _together_ with training (or fine tuning) scripts, and the entire evaluation pipeline to measure performance. And the data required to support all of this. When someone joins your data science team your would give them all this code and data. Not just the weights and say - the weights are the source, modify that to improve the model, I look forward to see yo…

It's not about the preferred way. Else open source software would need to give you their IDE setup, CI/CD setup, access to all internal tools, etc. Software like sqlite don't release their full test suite. They paywall the preferred way of making changes, yet they are open source.

>The “source code” for a work means the preferred form of the work for making modifications

The GPL refers to a form of the artifact being released

Re: OpenAI's new open-source model is basically Phi-5

#22

Post by wmf »

diggan wrote: post #19

> for instance, they have broad general knowledge about science, but don’t know much about popular culture That seems like a good focus. Why learn details that can change within days of it being released? Instead, train the models to have good general knowledge, and be really good at using tools, and you won't have to re-train models from scratch just because some JS library now has a different API, instead the model…

Yeah, it always seemed like a sad commentary on our world that AIs are devoting their weights to encyclopedic knowledge of Harry Potter, Pokemon, and Reddit trolling.

Re: OpenAI's new open-source model is basically Phi-5

#23

Post by jchw »

NitpickLawyer wrote: post #17
jchw wrote: post #10

Earlier quoted context omitted.

I think source code really only exists in terms of the source code/object code dichotomy, so what "traditional" open source means for model weights is really not obvious if you only go off of traditional definitions. Personally I think the word "open source" shouldn't apply here anymore than it would for art or binary code. Consider the following: it is possible to release binaries under the Apache2 license. Microsof…

> Consider the following: it is possible to release binaries under the Apache2 license. Microsoft has, at least at one point, released a binary under the BSD license. These binaries are not open source because they are not source. Agreed. But weights are not binaries in the licensing context. For weights to be binaries it would imply another layer of abstraction, above weights, that the labs use as the preferred way…

In my opinion, though, they're also not really source code either. They're an artifact of a training process, not code that was written by someone.

Re: OpenAI's new open-source model is basically Phi-5

#24

Post by charcircuit »

tuckerman wrote: post #5
NitpickLawyer wrote: post #2

Yeah, makes sense. Good observations regarding the benchmark vs. vibes in general, and I didn't know / made the connection between the lead of phi models going to oAI and gpt-oss. Could very well be a similar exercise + their "new" prompt level adherence (system > developer > user). In all the traces I've seen of refusals the model "quotes" the policy quite religiously. Similar thing was announced for gpt5. I think t…

I mostly agree with your assessment of what we should/shouldn't call open source for models but there is enough grey area to make the other side a valid position and not worthy of being dismissed so easily. I think there is a fine line between model weights and, say, bytecode for an interpreter and I think if you released bytecode dumps under any license it would be called out. I also believe the four freedoms are vi…

You are free to look at every single weight and study how it affects the result. You can see how the model is architected. And you don't need training data to be provided to be able to modify the weights. Software can still be open source even if it isn't friendly to beginners.

Re: OpenAI's new open-source model is basically Phi-5

#25

Post by kristopolous »

wmf wrote: post #20

I saw a bunch of people complaining on Twitter about how GPT-OSS can't be customized or has no soul and I noticed that none of them said what they were trying to accomplish. "The main use-case for fine-tuning small language models is for erotic role-play, and there’s a serious demand." Ah.

Porn is always the frontier.

It's a well-understood self-contained use-case without many externalities and simple business models.

What more, with porn, the medium is the product probably more than the content. Having it on home-media in the 80s was the selling point. Getting it over the 1-900 phone lines or accessing it over the internet ... these were arguably the actual product. It might have been a driver of early smart phone adoption as well. Adult content is about an 80% consumption on handheld devices while the internet writ large is about 60%.

Private tunable multi-media interaction on-demand is the product here.

Also it's a unique offer. Role playing prohibited sexual acts can be done arguably victim free.

There's a good fiction story there... "I thought I was talking to AI"

Re: OpenAI's new open-source model is basically Phi-5

#26

Post by izabera »

wmf wrote: post #20

I saw a bunch of people complaining on Twitter about how GPT-OSS can't be customized or has no soul and I noticed that none of them said what they were trying to accomplish. "The main use-case for fine-tuning small language models is for erotic role-play, and there’s a serious demand." Ah.

what's the problem with that? we have erotic texts dating back thousands of years, basically as old as the act of writing itself https://en.wikipedia.org/wiki/Istanbul_2461

Re: OpenAI's new open-source model is basically Phi-5

#27

Post by NitpickLawyer »

jchw wrote: post #23
NitpickLawyer wrote: post #17

Earlier quoted context omitted.

> Consider the following: it is possible to release binaries under the Apache2 license. Microsoft has, at least at one point, released a binary under the BSD license. These binaries are not open source because they are not source. Agreed. But weights are not binaries in the licensing context. For weights to be binaries it would imply another layer of abstraction, above weights, that the labs use as the preferred way…

In my opinion, though, they're also not really source code either. They're an artifact of a training process, not code that was written by someone.

> They're an artifact of a training process, not code that was written by someone.

If that were relevant to the licensing discussion, then you'd have to consider every "generated" parts (interfaces, dataclasses, etc) of every open source project artefacts. Historically, that was never the case. The license doesn't care if a hardcoded value was written by a person or "tuned" via a process. It's still source code if it's the preferred way of modifying said code. And it is. You can totally edit them by hand. It would not work as well (or at all), but you could do it.

Re: OpenAI's new open-source model is basically Phi-5

#28

Post by 127 »

NitpickLawyer wrote: post #14
mejutoco wrote: post #4

Earlier quoted context omitted.

The key is if you consider weights source code. I do not think this is a common interpretation. > The labs themselves modify models the same as you are allowed to by the license Do the labs do not use source code? It is a bit like arguing that releasing a binary executable is releasing the source code. One could claim developers modify the binary the same as you are allowed to.

> Do the labs do not use source code? The weights are part of the source code. When running inference on a model you use the architecture, config files and weights together. All of these are released. Weights are nothing but "hardcoded values". The way you reach those values is irrelevant in the license discussion. Let's take a simple example: I write a chess program that is comprised of a source file with 10 "if" st…

Training is obviously the compilation step.

Re: OpenAI's new open-source model is basically Phi-5

#29

Post by magicalhippo »

VladVladikoff wrote: post #18
magicalhippo wrote: post #15

I've found good use of Phi-4 at home, and after a few tests of the GPT-OSS 20B version I'm quite impressed so far. Particularly one SQL question that has tripped every other model of similar or smaller size that I've tried, like Devstral 24B, Falcon 3 7B, Qwen2.5-coder 14B and Phi 4 14B. The question contains an key point which is obvious for most humans, and which all of the models I tried previously have failed to…

Can you share the question? Or are you intentionally trying to keep it out of the training data pool?

Sadly no. I'd like to keep it untainted, but also because the tables involved are straight from my work, which is very much not OSS.

I can however try to paraphrase it so you get the gist of it.

The question asks to provide a SQL statement to update rows in table A based on related tables B and C, where table B is mentioned explicitly and C is implicit through the foreign keys provided in the context.

The key point all previous models I've tested has missed, is that the rows in A are many-to-one with B, and so the update should take this into account. This is implicit from the foreign key context and not mentioned directly in the question.

Think distributing pizza slices between a group of friends. All previous models has completely missed this part and just given each friend the whole pizza.

GPT-OSS correctly identified this issue and flagged it in the response, but also included a sensible assumption of evenly dividing the pizza.

I should note some of the previous models also missed the implicit connection to table C, and thus completely failed to do something sensible. But at least several of them figured this out. Of course I forgot to write that part down so can't say offhand which did what.

As for the code, for example I've coded a Y combinator in Delphi, using intentionally terse non-descriptive names, and asked the models to explain how the code works and what it does. Most ~7B models and larger of the past year or so have managed to explain it fairly well. However GPT-OSS was much more thorough and provider a much better explanation, showing a significantly better "understanding" of the code. It was also the first model smaller than LLama 3 70B that I've tried that correctly identified it as a Y combinator.

Re: OpenAI's new open-source model is basically Phi-5

#30

Post by philipkglass »

izabera wrote: post #26
wmf wrote: post #20

I saw a bunch of people complaining on Twitter about how GPT-OSS can't be customized or has no soul and I noticed that none of them said what they were trying to accomplish. "The main use-case for fine-tuning small language models is for erotic role-play, and there’s a serious demand." Ah.

what's the problem with that? we have erotic texts dating back thousands of years, basically as old as the act of writing itself https://en.wikipedia.org/wiki/Istanbul_2461

There's nothing wrong with it, but you have to understand the differences between different user groups to know which limitations are relevant to your own use cases. "It doesn't follow instructions" could mean "it won't pretend to be a horny elf" or "it hallucinates fields outside the JSON schema I specified"; the latter is much more of a problem for my uses.
Post reply on HN