Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Open source AI is the path forward

about.fb.com

Post reply on HN
Print view View original

381–390 of 936 posts

Re: Open source AI is the path forward

#381

Post by vbarrielle »

CamperBob2 wrote: post #347
Flimm wrote: post #164

Earlier quoted context omitted.

No, it's not. The Llama 3 Community License Agreement is not an open source license. Open source licenses need to meet the criteria of the only widely accepted definition of "open source", and that's the one formulated by the OSI [0]. This license has multiple restrictions on use and distribution which make it not open source. I know Facebook keeps calling this stuff open source, maybe in order to get all the good wi…

Open source licenses need to meet the criteria of the only widely accepted definition of "open source", and that's the one formulated by the OSI [0] Who died and made OSI God?

The OSI was created about 20 years ago and defined and popularized the term open source. Their definition has been widely accepted over that period.

Recently, companies are trying to market things as open source when in reality, they fail to adhere to the definition.

I think we should not let these companies change the meaning of the term, which means it's important to explain every time they try to seem more open than they are.

I'm afraid the battle is being lost though.

Re: Open source AI is the path forward

#382

Post by fractalf »

sheepscreek wrote: post #310
tambourine_man wrote: post #264

Earlier quoted context omitted.

Praising is good. Gratitude is a bit much. They got this big by selling user generated content and private info to the highest bidder. Often through questionable means. Also, the underdog always touts Open Source and standards, so it’s good to remain skeptical when/if tables turn.

All said and done, it is a very expensive and balsy way to undercut competitors. They’ve spent > $5B on hardware alone, much of which will depreciate in value quickly. Pretty sure the only reason Meta’s managed to do this is because of Zuck’s iron grip on the board (majority voting rights). This is great for Open Source and regular people though!

Well, he didn't do it to be "nice", you can be sure about that. Obviously they see a financial gain somewhere/sometime

Re: Open source AI is the path forward

#383

Post by phkahler »

Palomides wrote: post #364
phkahler wrote: post #302

Earlier quoted context omitted.

>> ...but this part strikes me as potentially anticompetitive reasoning. "Commoditize your complements" is an accepted strategy. And while pricing below cost to harm competitors is often illegal, the reality is that the marginal cost of software is zero.

spending a very quantifiable large amount of money to release something your nominal competitors charge for without having your own direct business case for it seems a little much

Companies spend very large amounts of money on all sorts of things that never even get released. Nothing wrong with releasing something for free that no longer costs you anything. Who knows why they developed it in the first place, it makes no difference.

Re: Open source AI is the path forward

#384

Post by downWidOutaFite »

input_sh wrote: post #170
the8thbit wrote: post #12

"Eventually though, open source Linux gained popularity – initially because it allowed developers to modify its code however they wanted ..." I find the language around "open source AI" to be confusing. With "open source" there's usually "source" to open, right? As in, there is human legible code that can be read and modified by the user? If so, then how can current ML models be open source? They're very large matric…

Open Source Initiative (kind of a de-facto authority on what's open source and what not) is spending a whole lot of time figuring out what it means for an AI system to be open source. In other words, they're basically trying to come up with a new license because the existing ones can't easily apply. I believe this is the current draft: https://opensource.org/deepdive/drafts/the-open-source-ai-de...

OSI made themselves the authority because they hated Richard Stallman and his Free Software movement. It's just marketing.

Re: Open source AI is the path forward

#385

Post by moffkalast »

cs702 wrote: post #335
tintor wrote: post #304

Earlier quoted context omitted.

> they have brought the cost of pretraining a frontier 400B+ parameter model to ZERO It is still far from zero.

If the model is already pretrained, there's no need to pretrain it, so the cost of pretraining is zero.

Yeah but you only have the one model, and so far it seems to be only good on paper.

Re: Open source AI is the path forward

#386

Post by KaiserPro »

rybosworld wrote: post #54

Huge companies like facebook will often argue for solutions that on the surface, seem to be in the public interest. But I have strong doubts they (or any other company) actually believe what they are saying. Here is the reality: - Facebook is spending untold billions on GPU hardware. - Facebook is arguing in favor of open sourcing the models, that they spent billions of dollars to generate, for free...? It follows th…

The model it's self isn't actually that valuable to facebook. The thing that's important is the dataset, the infrastructure and the people to make the models.

There is still, just about, a strong ethos( especially in the research teams) to chuck loads of stuff over the wall into opensource. (pytorch, detectron, SAM, aria etc)

but its seen internally as a two part strategy:

1) strong recruitment tool (come work with us, we've done cool things, and you'll be able to write papers)

2) seeding the research community with a common toolset.

Re: Open source AI is the path forward

#387

Post by moffkalast »

Zambyte wrote: post #238
the8thbit wrote: post #12

"Eventually though, open source Linux gained popularity – initially because it allowed developers to modify its code however they wanted ..." I find the language around "open source AI" to be confusing. With "open source" there's usually "source" to open, right? As in, there is human legible code that can be read and modified by the user? If so, then how can current ML models be open source? They're very large matric…

> If so, then how can current ML models be open source? The source of a language model is the text it was trained on. Llama models are not open source (contrary to their claims), they are open weight.

You can find the entire Llama 3.0 pretraining set here: https://huggingface.co/datasets/HuggingFaceFW/fineweb

15T tokens, 45 terrabytes. Seems fairly open source to me.

Re: Open source AI is the path forward

#388

Post by QuercusMax »

advael wrote: post #371
JKCalhoun wrote: post #356

Earlier quoted context omitted.

A good point. Forgive me, I am AI naive, is there some way to harness Llama to train ones own actually-open AI?

Kinda. Since you can self-host the model on a linux machine, there's no meaningful way for them to prevent you from having the trained weights. You can use this to bootstrap other models, or retrain on your own datasets, or fine-tune from the starting point of the currently-working model. What you can't do is be sure what they trained it on

How open is it really though? If you're starting from their weights, do you actually have legal permission to use derived models for commercial purposes? If it turns out that Meta used datasets they didn't have licenses to use in order to generate the model, then you might be in a big heap of mess.

Re: Open source AI is the path forward

#389

Post by thayne »

Kubuxu wrote: post #260
the8thbit wrote: post #161

Earlier quoted context omitted.

"There's a pretty clear difference between the 'finetuning' offered via API by GPT4 and the ability to do whatever sort of finetuning you want and get the weights at the end that you can do with open weights models." Yes, the difference is that one is provided over a remote API, and the provider of the API can restrict how you interact with it, while the other is performed directly by the user. One is a SaaS solution…

The gap between fine-tuning API and weights-available is much more significant than you give it credit for. You can take the weights and train LoRAs (which is close to fine-tuning), but you can also build custom adapters on top (classification heads). You can mix models from different fine-tunes or perform model surgery (adding additional layers, attention heads, MoE). You can perform model decomposition and amplify…

That still doesn't make it open source.

There is a massive difference between a compiled binary that you are allowed to do anything you want with, including modifying it, building something else on top or even pulling parts of it out and using in something else, and a SaaS offering where you can't modify the software at all. But that doesn't make the compiled binary open source.

Re: Open source AI is the path forward

#390

Post by holoduke »

advael wrote: post #324
cs702 wrote: post #74

> We’re releasing Llama 3.1 405B, the first frontier-level open source AI model, as well as new and improved Llama 3.1 70B and 8B models. Bravo! While I don't agree with Zuck's views and actions on many fronts, on this occasion I think he and the AI folks at Meta deserve our praise and gratitude. With this release, they have brought the cost of pretraining a frontier 400B+ parameter model to ZERO for pretty much ever…

Look, absolutely zero people in the world should trust any tech company when they say they care about or will keep commitments to the open-source ecosystem in any capacity. Nevertheless, it is occasionally strategic for them to do so, and there can be ancillary benefits for said ecosystem in those moments where this is the best play for them to harm their competitors For now, Meta seems to release Llama models in way…

In tech you can trust the underdogs. Once they turn into dominant players they turn evil. 99% of the cases.
Post reply on HN