Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Open source AI is the path forward

about.fb.com

Post reply on HN
Print view View original

171–180 of 936 posts

Re: Open source AI is the path forward

#171

Post by JumpCrisscross »

Palomides wrote: post #125
JumpCrisscross wrote: post #113

Earlier quoted context omitted.

> because it's really hard to do proper Open Source with these LLMs, means we need to change the meaning of Open Source so it fits with these PR releases? Open training data is hard to the point of impracticality. It requires excluding private and proprietary data. Meanwhile, the term "open source" is massively popular. So it will get used. The question is how. Meta et al would love for the choice to be between, on o…

source available is absolutely not the same as open source you are playing very loosely with terms that have specific, widely accepted definitions (e.g. https://opensource.org/osd ) I don't get why you think it would be useful to call LLMs with published weights "open source"

> terms that have specific, widely accepted definitions

OSF's definition is far from the only one [1]. Switzerland is currently implementing CH Open's definition, the EU another one, et cetera.

> I don't get why you think it would be useful to call LLMs with published weights "open source"

I don't. I'm saying that if the choice is between open weights or open weights + open training data, open weights will win because the useful definition will outcompete the pristine one in a public context.

[1] https://en.wikipedia.org/wiki/Open-source_software#Definitio...

Re: Open source AI is the path forward

#172

Post by unraveller »

diggan wrote: post #104

> Today we’re taking the next steps towards open source AI becoming the industry standard. We’re releasing Llama 3.1 405B, the first frontier-level open source AI model, Why do people keep mislabeling this as Open Source? The whole point of calling something Open Source is that the "magic sauce" of how to build something is publicly available, so I could built it myself if I have the means. But without the training d…

Open-weights is not open-source, for sure, but I don't mind it being stated as an aspiration goal, the moment it is legally possible to publish a source without shooting themselves in the foot they should do it.

They could release 50% of their best data but that would only stop them from attracting the best talent.

Re: Open source AI is the path forward

#173

Post by JeremyNT »

diggan wrote: post #104

> Today we’re taking the next steps towards open source AI becoming the industry standard. We’re releasing Llama 3.1 405B, the first frontier-level open source AI model, Why do people keep mislabeling this as Open Source? The whole point of calling something Open Source is that the "magic sauce" of how to build something is publicly available, so I could built it myself if I have the means. But without the training d…

> Why do people keep mislabeling this as Open Source?

I guess this is a rhetorical question, but this is a press release from Meta itself. It's just a marketing ploy, of course.

Re: Open source AI is the path forward

#174

Post by talldayo »

diggan wrote: post #151
valine wrote: post #135

Earlier quoted context omitted.

The codebase to do the training is way less valuable than the weights for the vast majority of people. Releasing the training code would be nice, but it doesn't really help anyone but Meta's direct competitors. If you want to train on top of Llama there's absolutely nothing stopping you. Plenty of open source tools to do parameter optimization.

Not just the training code but the training data as well, should be under a permissive license, otherwise you cannot call the project itself Open Source, which Facebook does here. > is way less valuable than the weights for the vast majority of people The same is true for most Open Source projects, most people use the distributed binaries or other artifacts from the projects, and couldn't care less about the code its…

I will steelman the idea that a tokenizer and weights are all you need for the "source" of an LLM. They are components that can be modified, redistributed and when put together, reproduce the full experience intended.

If we insist upon the release of training data with Open models, you might as well kiss the idea of usable Open LLMs out the door. Most of the content in training datasets like The Pile are not licensed for redistribution in any way shape or form. It would jeopardize projects that do use transparent training data while not offering anything of value to the community compared to the training code. Republishing all training data is an absolute trap.

Re: Open source AI is the path forward

#175

Post by chasd00 »

ketzo wrote: post #127
rybosworld wrote: post #54

Huge companies like facebook will often argue for solutions that on the surface, seem to be in the public interest. But I have strong doubts they (or any other company) actually believe what they are saying. Here is the reality: - Facebook is spending untold billions on GPU hardware. - Facebook is arguing in favor of open sourcing the models, that they spent billions of dollars to generate, for free...? It follows th…

Meta is, fundamentally, a user-generated-content distribution company. Meta wants to make sure they commoditize their complements: they don’t want a world where OpenAI captures all the value of content generation, they want the cost of producing the best content to be as close to free as possible.

i was thinking along the same. A lot of content generated by LLMs is going to end up on Facebook or Instagram. The easier it is to create AI generated content the more content ends up on those applications.

Re: Open source AI is the path forward

#176

Post by enriquto »

Nesco wrote: post #156
enriquto wrote: post #142

It's alarming that he refers to llama as if it was open source. The definition of free software (and open source, for that mater), is well-established. The same definition applies to all programs, whether they are "AI" or not. In any case, if a program was built by training against a dataset, the whole dataset is part of the source code. Llama is distributed in binary form, and it was built based on a secret dataset.…

The training data contains most likely insane amounts of copyrighted material. That’s why virtually none of the “open models” come with their training data

> The training data contains most likely insane amounts of copyrighted material.

If that is the case then the weights must inherit all these copyrights. It has been shown (at least in image processing) that you can extract many training images from the weights, almost verbatim. Hiding the training data does not solve this issue.

But regardless of copyright issues, people here are complaining about the malicious use of the term "open source", to signify a completely different thing (more like "open api").

Re: Open source AI is the path forward

#177

Post by skybrian »

FinchNova12 wrote: post #85

> Our adversaries are great at espionage, stealing models that fit on a thumb drive is relatively easy, and most tech companies are far from operating in a way that would make this more difficult. Mostly unrelated to the correctness of the article, but this feels like a bad argument. AFAIK, Anthropic/OpenAI/Google are not having issues with their weights being leaked (are they?). Why is it that Meta's model weights a…

I think it’s hard to say. We simply don’t know much from the outside. Microsoft has had some pretty bad security lapses, for example around guarding access to Windows source code. I don’t think we’ve seen a bad security break-in at Google in quite a few years? It would surprise me if Anthropic and OpenAI had good security since they’re pretty new, and fast-growing startups have a lot of organizational challenges.

It seems safe to assume that not all the companies doing leading-edge LLM’s have good security and that the industry as a whole isn’t set up to keep secrets for long. Things aren’t locked down to the level of classified research. And it sounds like Zuckerberg doesn’t want to play the game that way.

At the state level, China has independent AI research efforts and they’re going to figure it out. It’s largely a matter of timing, which could matter a lot.

There’s still an argument to be made against making proliferation too easy. Just because states have powerful weapons doesn’t mean you want them in the hands of people on the street.

Re: Open source AI is the path forward

#178

Post by kjkjadksj »

indus wrote: post #149

Is there an argument against Open Source AI? Not the usual nation-state rhetoric, but something that justifies that closed source leads to better user-experience and fewer security and privacy issues. An ecosystem that benefits vendors, customers, and the makers of close source? Are there historical analogies other than Microsoft Windows or Apple iPhone / iOS?

Lets take the iphone. Secured by the industries best security teams I am sure. Closed source, yet teenagers in eastern europe have cracked into it dozens of times making jailbreaks. Every law enforcement agency can crack into it. Closed source is not a security moat, but a trade protection moat.

Re: Open source AI is the path forward

#180

Post by jdminhbg »

enriquto wrote: post #142

It's alarming that he refers to llama as if it was open source. The definition of free software (and open source, for that mater), is well-established. The same definition applies to all programs, whether they are "AI" or not. In any case, if a program was built by training against a dataset, the whole dataset is part of the source code. Llama is distributed in binary form, and it was built based on a secret dataset.…

> In any case, if a program was built by training against a dataset, the whole dataset is part of the source code.

I'm not sure why I keep seeing this. What is the equivalent of the training data for something like the Linux kernel?

Post reply on HN