Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Open source AI is the path forward

about.fb.com

Post reply on HN
Print view View original

61–70 of 936 posts

Re: Open source AI is the path forward

#61

Post by skybrian »

jsheard wrote: post #29
the8thbit wrote: post #12

"Eventually though, open source Linux gained popularity – initially because it allowed developers to modify its code however they wanted ..." I find the language around "open source AI" to be confusing. With "open source" there's usually "source" to open, right? As in, there is human legible code that can be read and modified by the user? If so, then how can current ML models be open source? They're very large matric…

I also think that something like Chromium is a better analogy for corporate open source models than a grassroots project like Linux is. Chromium is technically open source, but Google has absolute control over the direction of it's development and realistically it's far too complex to maintain a fork without Googles resources, just like Meta has complete control over what goes into their open models, and even if they…

I think you’re right from the perspective of an individual developer. You and I are not about to fork Chromium any time soon. If you presume that forking is impractical then sure, the right to fork isn’t worth much.

But just because a single developer couldn’t do it doesn’t mean it couldn’t be done. It means nobody has organized a large enough effort yet.

For something like a browser, which is critical for security, you need both the organization and the trust. Despite frequent criticism, Mozilla (for example) is still considered pretty trustworthy in a way that an unknown developer can’t be.

Re: Open source AI is the path forward

#62

Post by causal »

mesebrec wrote: post #16

Note that Meta's models are not open source in any interpretation of the term. * You can't use them for any purpose. For example, the license prohibits using these models to train other models. * You can't meaningfully modify them given there is almost no information available about the training data, how they were trained, or how the training data was processed. As such, the model itself is not available under an op…

You are definitely allowed to train other models with these models, you just have to give credit in the name, per the license:

> If you use the Llama Materials or any outputs or results of the Llama Materials to create, train, fine tune, or otherwise improve an AI model, which is distributed or made available, you shall also include “Llama” at the beginning of any such AI model name.

Re: Open source AI is the path forward

#63

Post by tw04 »

JumpCrisscross wrote: post #20
tw04 wrote: post #17

>In the early days of high-performance computing, the major tech companies of the day each invested heavily in developing their own closed source versions of Unix. Because they sold the resultant code and systems built on it for money... this is the gold miner saying that all shovels and jeans should be free. Am I happy Facebook open sources some of their code? Sure, I think it's good for everyone. Do I think they're…

> if Facebook were selling their AI advances instead of selling the output it produces, they wouldn't be advocating for everyone else to open source their stack You're acting as if commoditizing one's complements is either new or reprehensible [1]. [1] https://gwern.net/complement

>You're acting as if commoditizing one's complements is either new or reprehensible [1].

I'm acting as if calling on other companies to open source their core product, just because it's a complement for you, and acting as if it's for the benefit of mankind is disingenuous, which it is.

Re: Open source AI is the path forward

#64

Post by ygjb »

CardenB wrote: post #22
JumpCrisscross wrote: post #7

“The Heavy Press Program was a Cold War-era program of the United States Air Force to build the largest forging presses and extrusion presses in the world.” This ”program began in 1944 and concluded in 1957 after construction of four forging presses and six extruders, at an overall cost of $279 million. Six of them are still in operation today, manufacturing structural parts for military and commercial aircraft” [1].…

Doubtful that GPUs purchased today would be in use for a similar time scale. Govt investment would also drive the cost of GPUs up a great deal. Not sure why a publicly accessible GPU cluster would be a better solution than the current system of research grants.

Of course they won't. The investment in the Heavy Press Program was the initial build, and just citing one example, the Alcoa 50,000 ton forging press was built in 1955, operated until 2008, and needed ~$100M to get it operational again in 2012.

The investment was made to build the press, which created significant jobs and capital investment. The press, and others like it, were subsequently operated by and then sold to a private operator, which in turn enabled the massive expansion of both military manufacturing, and commercial aviation and other manufacturing.

The Heavy Press Program was a strategic investment that paid dividends by both advancing the state of the art in manufacturing at the time it was built, and improving manufacturing capacity.

A GPU cluster might not be the correct investment, but a strategic investment in increasing, for example, the availability of training data, or interoperability of tools, or ease of use for building, training, and distributing models would probably pay big dividends.

Re: Open source AI is the path forward

#65

Post by Gracana »

causal wrote: post #50
1024core wrote: post #40

"open source AI" ... "open" ... "open" .... And you can't even try it without an FB/IG account. Zuck will never change.

I think you can use an HF account as well https://huggingface.co/meta-llama

You can also wait a bit for someone to upload quantized variants, finetunes, etc, and download those. FWIW I'm not making a claim about the legality of that, just saying it's an easy way around needing to sign the agreement.

Re: Open source AI is the path forward

#66

Post by Salgat »

rybosworld wrote: post #54

Huge companies like facebook will often argue for solutions that on the surface, seem to be in the public interest. But I have strong doubts they (or any other company) actually believe what they are saying. Here is the reality: - Facebook is spending untold billions on GPU hardware. - Facebook is arguing in favor of open sourcing the models, that they spent billions of dollars to generate, for free...? It follows th…

The reason for Meta making their model open source is rather simple: They receive an unimaginable amount of free labor, and their license only excludes their major competitors to ensure mass adoption without benefiting their competition (Microsoft, Google, Alibaba, etc). Public interest, philanthropy, etc are just nice little marketing bonuses as far as they're concerned (otherwise they wouldn't be including this licensing restriction).

Re: Open source AI is the path forward

#67

Post by tw04 »

rvnx wrote: post #35
tw04 wrote: post #17

>In the early days of high-performance computing, the major tech companies of the day each invested heavily in developing their own closed source versions of Unix. Because they sold the resultant code and systems built on it for money... this is the gold miner saying that all shovels and jeans should be free. Am I happy Facebook open sources some of their code? Sure, I think it's good for everyone. Do I think they're…

The source-code to Ad tracking platform is useless to users. At the end, it's actually Facebook doing the right thing (though they are known for being evil). It's a bit of an irony. The supposedly "good" and "open" people like Google or OpenAI, haven't given their model weights. A bit like Microsoft became the company that actually supports the whole open-source ecosystem with GitHub.

>The source-code to Ad tracking platform is useless to users.

It's absolutely not useless for developers looking to build a competing project.

>The supposedly "good" and "open" people like Google or OpenAI, haven't given their model weights.

Because they're monetizing it... the only reason Facebook is giving it away is because it's a complement to their core product of selling ads. If they were monetizing it, it would be closed source. Just like their Ads platform...

Re: Open source AI is the path forward

#69

Post by sidcool »

mvkel wrote: post #46

It's a real shame that we're still calling Llama "open source" when at best it's "open weights." Not that anyone would go buy 100,000 H100s to train their own Llama, but words matter. Definitions matter.

Honest question. As far as LLMs are concerned, isn't open weights same as open source?

Re: Open source AI is the path forward

#70

Post by mattnewton »

rybosworld wrote: post #54

Huge companies like facebook will often argue for solutions that on the surface, seem to be in the public interest. But I have strong doubts they (or any other company) actually believe what they are saying. Here is the reality: - Facebook is spending untold billions on GPU hardware. - Facebook is arguing in favor of open sourcing the models, that they spent billions of dollars to generate, for free...? It follows th…

I actually think this is one of the rare times where the small guys interests are aligned with Meta. Meta is scared of a world where they are locked out of LLM platforms, one where OpenAI gets to dictate rules around their use of the platform much like Apple and Google dictates rules around advertiser data and monetization on their mobile platforms. Small developers should be scared of a world where the only competitive LLMs are owned by those players too.

Through this lense, Meta’s actions make more sense to me. Why invest billions in VR/AR? The answer is simple, don’t get locked out of the next platform, maybe you can own the next one. Why invest in LLMs? Again, don’t get locked out. Google and OpenAi/Microsoft are far larger and ahead of Meta right now and Meta genuinely believes the best way to make sure they have an LLM they control is to make everyone else have an LLM they can control. That way community efforts are unified around their standard.

Post reply on HN