Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Open source AI is the path forward

about.fb.com

Post reply on HN
Print view View original

121–130 of 936 posts

Re: Open source AI is the path forward

#121

Post by bun_at_work »

Meta makes their money off advertising, which means they profit from attention.

This means they need content that will grab attention, and creating open source models that allow anyone to create any content on their own becomes good for Meta. The users of the models can post it to their Instagram/FB/Threads account.

Releasing an open model also releases Meta from the burden of having to police the content the model generates, once the open source community fine-tunes the models.

Overall, this move is good business move for Meta - the post doesn't really talk about the true benefit, instead moralizing about open source, but this is a sound business move for Meta.

Re: Open source AI is the path forward

#122

Post by whimsicalism »

the8thbit wrote: post #75
bilsbie wrote: post #26

Earlier quoted context omitted.

Can’t you do fine tuning on those binaries? That’s a modification.

You can fine tune the models, and you can modify binaries. However, there is no human readable "source" to open in either case. The act of "fine tuning" is essentially brute forcing the system to gradually alter the weights such that loss is reduced against a new training set. This limits what you can actually do with the model vs an actual open source system where you can understand how the system is working and mod…

I don't find this argument super convincing.

There's a pretty clear difference between the 'finetuning' offered via API by GPT4 and the ability to do whatever sort of finetuning you want and get the weights at the end that you can do with open weights models.

"Brute forcing" is not the correct language to use for describing fine-tuning. It is not as if you are trying weights randomly and seeing which ones work on your dataset - you are following a gradient.

Re: Open source AI is the path forward

#123

Post by JumpCrisscross »

mesebrec wrote: post #90
JumpCrisscross wrote: post #38

Earlier quoted context omitted.

> no one has bothered to release their training data If the FOSS community sets this as the benchmark for open source in respect of AI, they're going to lose control of the term. In most jurisdictions it would be illegal for the likes of Meta to release training data.

Regardless of the training data, the license even heavily restricts how you can use the model. Please read through their "acceptable use" policy before you decide whether this is really in line with open source.

> Please read through their "acceptable use" policy before you decide whether this is really in line with open source

I'm not taking a specific posiion on this license. I haven't read it closely. My broad point is simply that open source AI, as a term, cannot practically require the training data be made available.

Re: Open source AI is the path forward

#124

Post by plsbenice34 »

JumpCrisscross wrote: post #84
exe34 wrote: post #71

Earlier quoted context omitted.

the training data is the source.

> the training data is the source Sure. But that's not going to be released. The term open source AI cannot be expected to cover it because it's not practical.

Of course it could be practical - provide the data. The fact of that society is a dystopian nightmare controlled by a few megacorporations that don't want free information does not justify outright changing the meaning of the language.

Re: Open source AI is the path forward

#125

Post by Palomides »

JumpCrisscross wrote: post #113
diggan wrote: post #107

Earlier quoted context omitted.

So because it's really hard to do proper Open Source with these LLMs, means we need to change the meaning of Open Source so it fits with these PR releases?

> because it's really hard to do proper Open Source with these LLMs, means we need to change the meaning of Open Source so it fits with these PR releases? Open training data is hard to the point of impracticality. It requires excluding private and proprietary data. Meanwhile, the term "open source" is massively popular. So it will get used. The question is how. Meta et al would love for the choice to be between, on o…

source available is absolutely not the same as open source

you are playing very loosely with terms that have specific, widely accepted definitions (e.g. https://opensource.org/osd )

I don't get why you think it would be useful to call LLMs with published weights "open source"

Re: Open source AI is the path forward

#126

Post by resters »

This is really good news. Zuck sees the inevitability of it and the dystopian regulatory landscape and decided to go all in.

This also has the important effect of neutralizing the critique of US Government AI regulation because it will democratize "frontier" models and make enforcement nearly impossible. Thank you, Zuck, this is an important and historic move.

It also opens up the market to a lot more entry in the area of "ancillary services to support the effective use of frontier models" (including safety-oriented concerns), which should really be the larger market segment.

Re: Open source AI is the path forward

#127

Post by ketzo »

rybosworld wrote: post #54

Huge companies like facebook will often argue for solutions that on the surface, seem to be in the public interest. But I have strong doubts they (or any other company) actually believe what they are saying. Here is the reality: - Facebook is spending untold billions on GPU hardware. - Facebook is arguing in favor of open sourcing the models, that they spent billions of dollars to generate, for free...? It follows th…

Meta is, fundamentally, a user-generated-content distribution company.

Meta wants to make sure they commoditize their complements: they don’t want a world where OpenAI captures all the value of content generation, they want the cost of producing the best content to be as close to free as possible.

Re: Open source AI is the path forward

#128

Post by amelius »

> One of my [Mark Zuckerberg, ed.] formative experiences has been building our services constrained by what Apple will let us build on their platforms. Between the way they tax developers, the arbitrary rules they apply, and all the product innovations they block from shipping, it’s clear that Meta and many other companies would be freed up to build much better services for people if we could build the best versions of our products and competitors were not able to constrain what we could build.

This is hard to disagree with.

Re: Open source AI is the path forward

#130

Post by prpl »

JumpCrisscross wrote: post #7

“The Heavy Press Program was a Cold War-era program of the United States Air Force to build the largest forging presses and extrusion presses in the world.” This ”program began in 1944 and concluded in 1957 after construction of four forging presses and six extruders, at an overall cost of $279 million. Six of them are still in operation today, manufacturing structural parts for military and commercial aircraft” [1].…

Great idea, too bad the DOE and NSF were there first.
Post reply on HN