Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Open source AI is the path forward

about.fb.com

Post reply on HN
Print view View original

361–370 of 936 posts

Re: Open source AI is the path forward

#361

Post by logicchains »

ori_b wrote: post #346
advael wrote: post #324

Earlier quoted context omitted.

Look, absolutely zero people in the world should trust any tech company when they say they care about or will keep commitments to the open-source ecosystem in any capacity. Nevertheless, it is occasionally strategic for them to do so, and there can be ancillary benefits for said ecosystem in those moments where this is the best play for them to harm their competitors For now, Meta seems to release Llama models in way…

> If that ever stops being the case, you should fork rather than trust their judgment. Fork what? The secret sauce is in the training data and infrastructure. I don't think either of those is currently open.

They actually did open source the infrastructure library they developed. They don't open source the data but they describe how they gathered/filtered it.

Re: Open source AI is the path forward

#362

Post by tyler-jn »

cs702 wrote: post #74

> We’re releasing Llama 3.1 405B, the first frontier-level open source AI model, as well as new and improved Llama 3.1 70B and 8B models. Bravo! While I don't agree with Zuck's views and actions on many fronts, on this occasion I think he and the AI folks at Meta deserve our praise and gratitude. With this release, they have brought the cost of pretraining a frontier 400B+ parameter model to ZERO for pretty much ever…

So far, it seems like this release has done ~nothing to the stock price for GOOGL/MSFT, which we all know has been propped up largely on the basis of their AI plans. So it's probably premature to say that this has messed it up for them.

Re: Open source AI is the path forward

#363

Post by j_m_b »

> We need to protect our data.

This is a very important concern in Health Care because of HIPAA compliance. You can't just send your data over the wire to someone's proprietary API. You would at least need to de-identify your data. This can be a tricky task, especially with unstructured text.

Re: Open source AI is the path forward

#364

Post by Palomides »

phkahler wrote: post #302
6gvONxR4sf7o wrote: post #253

> Third, a key difference between Meta and closed model providers is that selling access to AI models isn’t our business model. That means openly releasing Llama doesn’t undercut our revenue, sustainability, or ability to invest in research like it does for closed providers. (This is one reason several closed providers consistently lobby governments against open source.) The whole thing is interesting, but this part…

>> ...but this part strikes me as potentially anticompetitive reasoning. "Commoditize your complements" is an accepted strategy. And while pricing below cost to harm competitors is often illegal, the reality is that the marginal cost of software is zero.

spending a very quantifiable large amount of money to release something your nominal competitors charge for without having your own direct business case for it seems a little much

Re: Open source AI is the path forward

#366

Post by 8note »

Flimm wrote: post #164
mkolodny wrote: post #73

Earlier quoted context omitted.

Llama’s code is open source: https://github.com/meta-llama/llama3/blob/main/llama/model.p...

No, it's not. The Llama 3 Community License Agreement is not an open source license. Open source licenses need to meet the criteria of the only widely accepted definition of "open source", and that's the one formulated by the OSI [0]. This license has multiple restrictions on use and distribution which make it not open source. I know Facebook keeps calling this stuff open source, maybe in order to get all the good wi…

Isn't the MIT license the generally accepted "open source" license? It's a community owned term, not OSI owned

Re: Open source AI is the path forward

#367

Post by advael »

quasse wrote: post #355
ori_b wrote: post #346

Earlier quoted context omitted.

> If that ever stops being the case, you should fork rather than trust their judgment. Fork what? The secret sauce is in the training data and infrastructure. I don't think either of those is currently open.

I'm just a lowly outsider to the AI space, but calling these open source models seems kind of like calling a compiled binary open source. If you don't have a way to replicate what they did to create the model, it seems more like freeware than open source.

As an ML researcher, I agree. Meta doesn't include adequate information to replicate the models, and from the perspective of fundamental research, the interest that big tech companies have taken in this field has been a significant impediment to independent researchers, despite the fact that they are undeniably producing groundbreaking results in many respects, due to this fundamental lack of openness

This should also make everyone very skeptical of any claim they are making, from benchmark results to the legalities involved in their training process to the prospect of future progress on these models. Without being able to vet their results against the same datasets they're using, there is no way to verify what they're saying, and the credulity that otherwise smart people have been exhibiting in this space has been baffling to me

As a developer, if you have a working Llama model, including the source code and weights, and it's crucial for something you're building or have already built, it's still fundamentally a good thing that Meta isn't gating it behind an API and if they went away tomorrow, you could still use, self-host, retrain, and study the models

Re: Open source AI is the path forward

#368

Post by gpm »

hluska wrote: post #313
gpm wrote: post #236

Earlier quoted context omitted.

You can't just look at the costs to an action, you also have to look at the benefits. Of course I agree I'm going to stop marginal investments from occurring into research into patent-able technologies by reducing the expect profit. But I'm going to do so very slightly because I'm not shifting the expected value by very much. Meanwhile I'm going to greatly increase the investment into the existing technology we alrea…

You have proposed state ownership of all successful IP. That is a massive change and yet you have demonstrated zero understanding of the possible costs. Your claim that removing a profit motivation will increase investment is flat out wrong. Everything else crumbles from there.

No, I've proposed removing or reducing IP protections, not transferring them to the state. Allowing competitors to enter the market will obviously increase investment in competitors...

Re: Open source AI is the path forward

#369

Post by Sparkyte »

The real path forward is recognizing what AI is good at and what it is bad at. Focus on making what it is good at even better and faster. Open AI will definitely give us that option but it isn't a miracle worker.

My impression is that AI if done correctly will be the new way to build APIs with large data sets and information. It can't write code unless you want to dump billions of dollars into a solution with millions of dollars of operational costs. As it stands it loses context too quickly to do advance human tasks. BUT this is where it is great at assembling data and information. You know what is great at assembling data and information? APIs.

Think of it this way if we can make it faster and it trains on a datalake for a company it could be used to return information faster than a nested micro-service architecture that is just a spiderweb of dependencies.

Because AI loses context simple API requests could actually be more efficient.

Re: Open source AI is the path forward

#370

Post by bornfreddy »

the8thbit wrote: post #144
stale2002 wrote: post #82

Earlier quoted context omitted.

Ok call it Open Weights then if the dictionary definitions matter so much to you. The actual point that matters is that these models are available for most people to use for a lot of stuff, and this is way way better than what competitors like OpenAI offer.

They don't "[allow] developers to modify its code however they want", which is a critical component of "open source", and one that Meta is clearly trying to leverage in branding around its products. I would like them to start calling these "public weight models", because what they're doing now is muddying the waters so much that "open source" now just means providing an enormous binary and an open source harness to r…

"Public weight models" sounds about right, thanks for coming up with a good term! Hope it catches.
Post reply on HN