Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

If you can't reproduce the model then it's not open-source

twitter.com

Post reply on HN
Print view View original

31–40 of 117 posts

Re: If you can't reproduce the model then it's not open-source

#31

Post by declaredapple »

fzliu wrote: post #9
ssgodderidge wrote: post #2

> Imagine if Linux published only a binary without the codebase. Or published the codebase without the compiler used to make the binary. This is where we are today. This was such a helpful way to frame the problem! Something felt off about the "open source models" out there; this highlights the problem incredibly well.

In my mind, what's more crucial here is code for downloading/scraping and labeling the data, not the model architecture nor training script. As much as I appreciate Mis(x)tral, I would've loved it even more if they released code for gathering data.

I'm speculating they are attempting to avoid controversy about their datasources. That and a possible competitive edge depending on what specific sets/filtering they're using.

Re: If you can't reproduce the model then it's not open-source

#32

Post by kube-system »

zelon88 wrote: post #25
stevehine wrote: post #24

Earlier quoted context omitted.

This depends, how many times does it need to hear the song to build up a reasonably consistent internal reproduction, and are you paying per stream or buying the input data as CD Singles - or just putting the AI in a room with the radio on and waiting for it to take in the playlist a few times ?

Let's assume it is in a room with a radio listening to music, and that the AI is "general purpose" meaning that it can also perform other functions. It is not the sole purpose of the AI to do this all day. I see where you are coming from in trying to identify the source of the copyright. This would be important information if a human wanted to sue another human for re-producing copyright material. However, does that…

The standards applied are exactly the same regardless of what tools are used. It doesn't matter if you're talking about a dumb AI, a general purpose AI, or a Xerox machine.

If you want an exception to copyright, you're going to want to start looking at a section 107 (of the copyright act) exception: https://www.copyright.gov/title17/92chap1.html#107

The reason someone walking down the street and humming a song is not a violation is because it very clearly meets all of the tests in section 107.

The biggest problem with feeding stuff through a black box like an LLM is it isn't easy for a human to determine how close the result is to the original. An LLM could act like a Xerox machine, and it won't tell you.

Re: If you can't reproduce the model then it's not open-source

#33

Post by ssgodderidge »

declaredapple wrote: post #31
fzliu wrote: post #9

Earlier quoted context omitted.

In my mind, what's more crucial here is code for downloading/scraping and labeling the data, not the model architecture nor training script. As much as I appreciate Mis(x)tral, I would've loved it even more if they released code for gathering data.

I'm speculating they are attempting to avoid controversy about their datasources. That and a possible competitive edge depending on what specific sets/filtering they're using.

To avoid controversy AND potential lawsuits.

Re: If you can't reproduce the model then it's not open-source

#34

Post by declaredapple »

I'll venture to say the majority of these "open access models" are meant to serve as advertisements of capabilities (either of hardware, research, or techniques) and nothing more. MPT being one of the most obvious example.

Many don't offer any information, some do offer information but provide no new techniques and just threw a bunch of compute and some data to make a sub-par model that shows up on a specific leaderboard.

Everyone is trying to save a card up their sleeve so they can sell it. And showing up on scoreboards is a great advertisement.

Re: If you can't reproduce the model then it's not open-source

#35

Post by declaredapple »

ssgodderidge wrote: post #33
declaredapple wrote: post #31

Earlier quoted context omitted.

I'm speculating they are attempting to avoid controversy about their datasources. That and a possible competitive edge depending on what specific sets/filtering they're using.

To avoid controversy AND potential lawsuits.

Yup.

I think many countries (japan already has) will allow IP for training data.

They just need to buy time until then.

Re: If you can't reproduce the model then it's not open-source

#37

Post by ivan_gammel »

RcouF1uZ4gsC wrote: post #10

I would argue that while technically correct, it is not what most people really care. What they care about are the following: 1. Can I download it? 2. Can I run it on my hardware? 3. Can I modify it? 4. Can I share my modifications with others? If those questions are in the affirmative, then I think most people consider it open enough, and it is a huge step for freedom compared to the models such as OpenAI.

„Can it be trusted?“ is the question many people will care about, when the awareness of the risks becomes higher. If this question can be answered without publishing the source, fine, but that would probably mean that publisher must be liable for damages from model output.

Re: If you can't reproduce the model then it's not open-source

#38

Post by tqi »

"the project does not benefit from the OSS feedback loop" It's not like you can submit PRs to training data that fixes specific issues the way you can submit bug fixes, so I'm skeptical you would see much of a feedback loop.

"it’s hard to verify that the model has no backdoors (eg sleeper agents)" Again given the size of the datasets and the opaque way training works, I am skeptical that anyone would be able tell if there is a backdoor in the training data.

"impossible to verify the data and content filter and whether they match your company policy" I don't totally know what this means. For one, you can/probably should apply company policies to the model outputs, which you can do without access to training data. Is the idea that every company could/should filter input data and train their own models?

"you are dependent on the company to refresh the model" At the current cost, this is probably already true for most people.

"A true open-source LLM project — where everything is open from the codebase to the data pipeline — could unlock a lot of value, creativity, and improve security." I am overall skeptical that this is true in the case of LLMs. If anything, I think this creates a larger surface for bad actors to attack.

Re: If you can't reproduce the model then it's not open-source

#39

Post by FooBarWidget »

ssgodderidge wrote: post #2

> Imagine if Linux published only a binary without the codebase. Or published the codebase without the compiler used to make the binary. This is where we are today. This was such a helpful way to frame the problem! Something felt off about the "open source models" out there; this highlights the problem incredibly well.

I think a better analogy is firmware binary blobs in the Linux kernel, or VM bytecodes.

The LLM inference engine (architecture implementation) is like a kernel driver that loads a firmware binary blob, or a virtual machine that loads bytecode. The inference engine is open source. The problem is that the weights (firmware blobs, VM bytecodes) are opaque: you don't have the means to reproduce them.

The Linux community has long argued that drivers that load firmware blobs are cheating: they don't count as open source.

Still, the "open source" LLMs are more open than "API-gated" LLMs. It's a step in the right direction, but I hope we don't stop there.

Re: If you can't reproduce the model then it's not open-source

#40

Post by zelon88 »

kube-system wrote: post #32
zelon88 wrote: post #25

Earlier quoted context omitted.

Let's assume it is in a room with a radio listening to music, and that the AI is "general purpose" meaning that it can also perform other functions. It is not the sole purpose of the AI to do this all day. I see where you are coming from in trying to identify the source of the copyright. This would be important information if a human wanted to sue another human for re-producing copyright material. However, does that…

The standards applied are exactly the same regardless of what tools are used. It doesn't matter if you're talking about a dumb AI, a general purpose AI, or a Xerox machine. If you want an exception to copyright, you're going to want to start looking at a section 107 (of the copyright act) exception: https://www.copyright.gov/title17/92chap1.html#107 The reason someone walking down the street and humming a song is not…

I think this conversion has corrected some misgivings I had about the AI copyright argument. My takeaway is;

Possession copyright material is not inherently infringing on a copyright. Disseminating copyright material is unless you meet section 107. AI runs afoul of section 107 when it verbatim shares copyright material from its dataset without attribution.

Post reply on HN