Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI's new open-source model is basically Phi-5

seangoedecke.com

Post reply on HN
Print view View original

31–40 of 233 posts

Re: OpenAI's new open-source model is basically Phi-5

#31

Post by j_timberlake »

wmf wrote: post #20

I saw a bunch of people complaining on Twitter about how GPT-OSS can't be customized or has no soul and I noticed that none of them said what they were trying to accomplish. "The main use-case for fine-tuning small language models is for erotic role-play, and there’s a serious demand." Ah.

You don't understand! Every erotic chatbot service keeps getting censored, what happened to CharacterAI just keeps happening. There's a serious supply-shortage, do you really want people turning to Grok? The spice must flow!!!

Re: OpenAI's new open-source model is basically Phi-5

#32

Post by shortrounddev2 »

kristopolous wrote: post #25
wmf wrote: post #20

I saw a bunch of people complaining on Twitter about how GPT-OSS can't be customized or has no soul and I noticed that none of them said what they were trying to accomplish. "The main use-case for fine-tuning small language models is for erotic role-play, and there’s a serious demand." Ah.

Porn is always the frontier. It's a well-understood self-contained use-case without many externalities and simple business models. What more, with porn, the medium is the product probably more than the content. Having it on home-media in the 80s was the selling point. Getting it over the 1-900 phone lines or accessing it over the internet ... these were arguably the actual product. It might have been a driver of earl…

There's something Freudian about the idea that the more you can customize porn, the more popular it is. That, despite the impression that "all men want one thing", it turns out that men all want very different and very oddly specific things. Imbuing somrthing with a "magical" quality that doesnt exist is the origin of the term "fetish". Its not about the raw attractive preference for a particular hair color; its a belief in the POWER of that hair color.

Re: OpenAI's new open-source model is basically Phi-5

#33

Post by wmf »

izabera wrote: post #26
wmf wrote: post #20

I saw a bunch of people complaining on Twitter about how GPT-OSS can't be customized or has no soul and I noticed that none of them said what they were trying to accomplish. "The main use-case for fine-tuning small language models is for erotic role-play, and there’s a serious demand." Ah.

what's the problem with that? we have erotic texts dating back thousands of years, basically as old as the act of writing itself https://en.wikipedia.org/wiki/Istanbul_2461

I have no problem with it and I can understand why people don't want to say "I'm trying to pornify this model and it refuses to talk dirty!" in public. But if you're calling a model garbage maybe you should be honest about what the "problem" is.

Re: OpenAI's new open-source model is basically Phi-5

#34

Post by jdiff »

NitpickLawyer wrote: post #14
mejutoco wrote: post #4

Earlier quoted context omitted.

The key is if you consider weights source code. I do not think this is a common interpretation. > The labs themselves modify models the same as you are allowed to by the license Do the labs do not use source code? It is a bit like arguing that releasing a binary executable is releasing the source code. One could claim developers modify the binary the same as you are allowed to.

> Do the labs do not use source code? The weights are part of the source code. When running inference on a model you use the architecture, config files and weights together. All of these are released. Weights are nothing but "hardcoded values". The way you reach those values is irrelevant in the license discussion. Let's take a simple example: I write a chess program that is comprised of a source file with 10 "if" st…

If you have 10 harcoded values, you have a binary blob, a common feature particularly in hardware drivers that is opaque and commonly considered to not be fully free unless the instructions for deriving it are also included. It's frequently just an executable, occasionally just configuration information, but difficult to change while (assuming no signing shenanigans) still remaining technically possible.

The training data is the source code and the training process is the compiler. There's a fairly direct metaphor to be made there. Different compilers can have vastly different impacts on the performance of the compiled executable.

Re: OpenAI's new open-source model is basically Phi-5

#35

Post by kristopolous »

shortrounddev2 wrote: post #32
kristopolous wrote: post #25

Earlier quoted context omitted.

Porn is always the frontier. It's a well-understood self-contained use-case without many externalities and simple business models. What more, with porn, the medium is the product probably more than the content. Having it on home-media in the 80s was the selling point. Getting it over the 1-900 phone lines or accessing it over the internet ... these were arguably the actual product. It might have been a driver of earl…

There's something Freudian about the idea that the more you can customize porn, the more popular it is. That, despite the impression that "all men want one thing", it turns out that men all want very different and very oddly specific things. Imbuing somrthing with a "magical" quality that doesnt exist is the origin of the term "fetish". Its not about the raw attractive preference for a particular hair color; its a be…

oh it's wildly different. About 15 years ago I worked on a porn recommendation system. The idea is that you'd follow a number of sites based on likes and recommendations and you'd get an aggregated feed with interstitial ads.

So I started with scraping and cross-reference, foaf, doing analysis. People's preferences are ... really complex.

Without getting too lewd, let's say there's about 30-80 categories with non-marginal demand depending on how you want to slice it and some of them can stack so you get a combinatoric.

In early user testing people wanted the niche and found the adventurous (of their particular kind) to be more compelling. And that was the unpredictable part. The majoritarian categories didn't have stickiness.

Nor did these niches have high correlation. Someone could be into say, specific topic A (let's say feet), and correlating that with topic B (let's say leather) was a dice roll. The probabilities were almost universally People want adventure on a reservation with a very well defined perimeter - one that is hard to map and different for every person.

So the value-add proposition went away since it's now just a collection of niche sites again.

Also, these days people have Reddit accounts reserved for porn where they do exactly this. So it was built after all.

Re: OpenAI's new open-source model is basically Phi-5

#36

Post by tuckerman »

charcircuit wrote: post #24
tuckerman wrote: post #5

Earlier quoted context omitted.

I mostly agree with your assessment of what we should/shouldn't call open source for models but there is enough grey area to make the other side a valid position and not worthy of being dismissed so easily. I think there is a fine line between model weights and, say, bytecode for an interpreter and I think if you released bytecode dumps under any license it would be called out. I also believe the four freedoms are vi…

You are free to look at every single weight and study how it affects the result. You can see how the model is architected. And you don't need training data to be provided to be able to modify the weights. Software can still be open source even if it isn't friendly to beginners.

I think you could say something remarkably similar about just releasing bytecode as well and I think most people would call foul at that. I don't think it's so cut and dry.

This isn't entirely about being a beginner or not either. Full fine-tuning without forgetting does really want the training data (or something that is a good replacement). You can do things like LoRa but, depending on your use case, it might not work.

Re: OpenAI's new open-source model is basically Phi-5

#37

Post by tuatoru »

kristopolous wrote: post #25
wmf wrote: post #20

I saw a bunch of people complaining on Twitter about how GPT-OSS can't be customized or has no soul and I noticed that none of them said what they were trying to accomplish. "The main use-case for fine-tuning small language models is for erotic role-play, and there’s a serious demand." Ah.

Porn is always the frontier. It's a well-understood self-contained use-case without many externalities and simple business models. What more, with porn, the medium is the product probably more than the content. Having it on home-media in the 80s was the selling point. Getting it over the 1-900 phone lines or accessing it over the internet ... these were arguably the actual product. It might have been a driver of earl…

1, Porn. 2 Military.

Re: OpenAI's new open-source model is basically Phi-5

#38

Post by jchw »

NitpickLawyer wrote: post #27
jchw wrote: post #23

Earlier quoted context omitted.

In my opinion, though, they're also not really source code either. They're an artifact of a training process, not code that was written by someone.

> They're an artifact of a training process, not code that was written by someone. If that were relevant to the licensing discussion, then you'd have to consider every "generated" parts (interfaces, dataclasses, etc) of every open source project artefacts. Historically, that was never the case. The license doesn't care if a hardcoded value was written by a person or "tuned" via a process. It's still source code if it…

There is actually a gray area about what code "counts" as source code to the point where you would consider it "open source" if it were licensed as such. I think if you had a repository consisting of only generated code and not the code used to generate it, it would definitely raise the question of whether it should be considered "source code" or "open source", and I think you could make arguments both ways.

On the other hand, I don't really think that argument then extends to model weights, which are not just some number of steps removed from source code, but just simply not really related to source code.

Re: OpenAI's new open-source model is basically Phi-5

#39

Post by michaelt »

izabera wrote: post #26
wmf wrote: post #20

I saw a bunch of people complaining on Twitter about how GPT-OSS can't be customized or has no soul and I noticed that none of them said what they were trying to accomplish. "The main use-case for fine-tuning small language models is for erotic role-play, and there’s a serious demand." Ah.

what's the problem with that? we have erotic texts dating back thousands of years, basically as old as the act of writing itself https://en.wikipedia.org/wiki/Istanbul_2461

The pro-porn side has zero PR because respectable public figures don't see pro-porn advocacy as a good career move. At most, you'll get some oblique references to it.

Meanwhile, the anti-porn side has a formidable alliance:

Right-wing, religiously-motivated anti-porn activists. Left-wing, feminism-motivated anti-porn activists. Big corporate types with lots of $$$$ to spend who want their customer support chatbot to be completely SFW at all times. AI safety folk who think keeping the model on a tight leash is an ethical obligation, lest future iterations take over the world. AI vendors who are keen on the yes-it-might-take-over-the-world narrative. AI vendors who just don't want their developers having to handle NSFW stuff in work. Politicians who don't know a transformer from a diffusion model, but who've heard a chorus of worries about lost jobs and AI bias and deepfakes and revenge porn.

These people will speak up in public at the drop of a hat.

Re: OpenAI's new open-source model is basically Phi-5

#40

Post by dullcrisp »

philipkglass wrote: post #30
izabera wrote: post #26

Earlier quoted context omitted.

what's the problem with that? we have erotic texts dating back thousands of years, basically as old as the act of writing itself https://en.wikipedia.org/wiki/Istanbul_2461

There's nothing wrong with it, but you have to understand the differences between different user groups to know which limitations are relevant to your own use cases. "It doesn't follow instructions" could mean "it won't pretend to be a horny elf" or "it hallucinates fields outside the JSON schema I specified"; the latter is much more of a problem for my uses.

    {
      "race": "elf",
      "horny": false
      ^^^^^^^^^^^^^^
      Unsupported value.
Post reply on HN