Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Meta to release open-source commercial AI model

zdnet.com

Post reply on HN
Print view View original

121–130 of 168 posts

Re: Meta to release open-source commercial AI model

#121

Post by briancleland »

doctoboggan wrote: post #80
ssd532 wrote: post #73

Earlier quoted context omitted.

I read it in all such discussions. What does it mean? I just have a very high level understanding of AI models. No idea how things work under the hood or what knobs can be tweaked.

The source code is all the supporting code needed to run inference on the weights. This is usually python and in the case of llama it's already open source. Usually the source code is referred to as the "model". You can kind of think of the weights as a settings file in a normal desktop application. The desktop app has its own source code and loads in the settings file at runtime. It can load different settings files…

This is almost completely wrong. When peope who work in AI refer to the "model", they are generally referring to the weights. It is the weights which are the most important determinant of how the model performs, and it is the weights that require the most resources to develop. Associated code and other assets are also important, but they not the core asset. The intuitive sense of open sourcing a model therefore typically means releasing the weights under an open licence (ideally along with the training and inference code, data, training info, etc).

Re: Meta to release open-source commercial AI model

#122

Post by axus »

zargon wrote: post #50
spmurrayzzz wrote: post #42

Earlier quoted context omitted.

Maybe this is just semantics, but I don't know if the OSS-vs-freemium distinction matters all that much (I'd have to think about the potential downsides a bit more tbh). Virtually every discussion in the LLM space right now is almost immediately bifurcated by the "can I use this commercially?" question which has a somewhat chilling effect on innovation. The best performing open source LLMs we have today are llama-bas…

Llama isn't open source either. But if I understand your point correctly, you're saying that the commercial use axis is what is important to people, and it's orthogonal to freeware vs open source. In the present environment, I agree. But I don't think we should let companies get away with poisoning the term open source for things which are not. I also believe that actual open source models have the near-term opportun…

Sen. Marsha Blackburn said “fair use” protections have become a “fairly useful way to steal” intellectual property. Some people would like to use this situation to get rid of "fair use".

Re: Meta to release open-source commercial AI model

#123

Post by smoldesu »

hospitalJail wrote: post #40
jerrygenser wrote: post #30

Earlier quoted context omitted.

Based on the podcast with Lex Friedman and Mark Zuckerberg, see ~minute 30. My hypothesis based on the context of Mark discussing the release is that it's going to be completely open source and can licensed to be used commercially. Not that Meta is going to add a whole new revenue side of business to compete with OpenAI. i.e. "Here is model, with commercially permissive licensing" not "Here is model that you can use…

Another hypothesis is that they are trying to rehab their brand. They can even write it as 'good will' on their financial statements. It kind of is working.

This isn't some recent revelation or anything. Facebook's AI team (FAIR) open sourced their major technology in 2017 with Pytorch. In 2018 they published Pytext in an age when most people didn't know what a Large Language Model even meant. Seeing LLaMA get made should not be a surprise to anyone who is familiar with the history of AI research. It's like hearing people call CUDA an "unfair advantage" while ignoring billions of Nvidia R&D dollars getting spent in the AI sector over the course of a decade.

It might feel like "brand rehab" or "good will" as a consumer, but a lot of this work was put in motion a while ago.

Re: Meta to release open-source commercial AI model

#124

Post by [deleted] »

roody15 wrote: post #64
strikelaserclaw wrote: post #55

Earlier quoted context omitted.

kind of a dystopian nightmare world in which large corporations utilize AI to create low cost, infinite content that humans engage with (mostly content catering to the human tendency for tribalism, prestige, sexual desires etc...), sounds like we are creating a world similar to the Matrix.

I think we may have already entered it. Infinite scroll based feeds like TikTok, Instagram, and Threads (and possible Reddit these days) … just AI algorithm deciding what you should find “entertaining” or “important”. It’s really the ultimate nightmare with the internet becoming just TV 3.0 in which content is controlled and curated … you just consume mindlessly. Any attempts to create a Reddit clone.. or system in w…

[deleted]

Re: Meta to release open-source commercial AI model

#125

Post by nomel »

strikelaserclaw wrote: post #55
jonnat wrote: post #41

Earlier quoted context omitted.

Quite the opposite, this is great for Meta's competitors. Meta is not trying to get market share with this strategy, it's trying to commoditize their complements ( https://www.joelonsoftware.com/2002/06/12/strategy-letter-v/ ) Content is a complement to a social network: the cheaper it is to create content, the more content is available, the easier it is to optimize a feed, the larger the time people spend in the pla…

kind of a dystopian nightmare world in which large corporations utilize AI to create low cost, infinite content that humans engage with (mostly content catering to the human tendency for tribalism, prestige, sexual desires etc...), sounds like we are creating a world similar to the Matrix.

https://en.wikipedia.org/wiki/Infinite_Jest

> ... Infinite Jest, also called "the Entertainment" or "the samizdat". The film is so compelling that its viewers lose all interest in anything other than repeatedly viewing it, and thus eventually die.

Re: Meta to release open-source commercial AI model

#126

Post by twayt »

xnx wrote: post #86
twayt wrote: post #78

Earlier quoted context omitted.

> They probably can: No, actually they probably can’t. There is no verifiable way to remove the data from the model apart from completely removing all instances of information from the training data. The project you linked only describes a selective finetuning approach.

It's an area of active research: https://ai.googleblog.com/2023/06/announcing-first-machine-u...

Until you get models with completely disentangled feature spaces such that you know that the influence of a piece of data is completely removed (at the limit this is something like an embedding DB), there is absolutely no way you can claim you’ve removed the data from the model.

At most, these efforts will amount to data laundering where it will be impossible to prove that a piece of data was used to train the model, not provide conclusive proof that it was removed.

Re: Meta to release open-source commercial AI model

#127

Post by freedomben »

pmarreck wrote: post #66

I have a 128 core Threadripper, a 2080 Ti and a 3080 Ti. How can I play with open source LLM's locally?

May I ask why you have such an amazing machine, and two nice graphics cards? Feel free to tell me it's none of my business, it's just very interesting to me :-)

Re: Meta to release open-source commercial AI model

#129

Post by Auracle »

411111111111111 wrote: post #99
TX81Z wrote: post #71

Earlier quoted context omitted.

The production of knowledge needs to be funded as it isn’t “free”. Copyright and licensing is one model that has worked for a long time. It has flaws, but it has produced good things. At this point with the quality of current web content and the collapse of journalism as an industry I think we can say online ads have utterly failed as a replacement income stream. Unless you want all LLM to say “I’m sorry the data I w…

You state this as a fact, but it's actually much less certain wherever it's ever been net-positive. It was probably intended that way, but the reality is that the power has been with the publisher since the beginning, and they've absolutly been screwing over the author's as well. Only the most successful author's have gotten decent deals. I don't have an answer to this either though, i just wanted to point out that c…

The only way you’d know is to A/B test with a country with no copyright, and see how their authors get by.

My guess is extremely poorly. Again, the biggest might be fine. Instead of publishers paying fairly little to authors they could just literally take the best books and print them, taking all of the profits…not to mention ebooks.

I’m not an author so I can’t speak to how much publishers make, but I’d assume that if one was way better than the others in how much they’re distribute to authors all of the best authors would jump ship. Markets have a way of working things out.

A lot of people want to be authors, and any time that happens - game dev, teachers, musicians, etc. - you’re going to take on a bit of extra hardship compared to other jobs.

Re: Meta to release open-source commercial AI model

#130

Post by Vetch »

zargon wrote: post #50
spmurrayzzz wrote: post #42

Earlier quoted context omitted.

Maybe this is just semantics, but I don't know if the OSS-vs-freemium distinction matters all that much (I'd have to think about the potential downsides a bit more tbh). Virtually every discussion in the LLM space right now is almost immediately bifurcated by the "can I use this commercially?" question which has a somewhat chilling effect on innovation. The best performing open source LLMs we have today are llama-bas…

Llama isn't open source either. But if I understand your point correctly, you're saying that the commercial use axis is what is important to people, and it's orthogonal to freeware vs open source. In the present environment, I agree. But I don't think we should let companies get away with poisoning the term open source for things which are not. I also believe that actual open source models have the near-term opportun…

Neural network weights are better viewed as source code because they specify what function the network computes. As we're operating purely on feed-forward networks, there are no loops. Therefore, weights fully describe everything relevant for executing their represented function on inputs. Weights can be seen as a sort of intermediate language (with lots of stored data and partially computed states) interpretable by some deep learning library.

The network architecture itself is not source code, but a rough specification constraining the optimizer, which searches for possible program descriptions that within the specified constraints, minimize some loss function with respect to the data.

Neither data nor network architecture are the actual source, they are better seen as recipes which if followed (will at great expense), allow finding behaviorally similar programs. As you can see, the standard ideas of open source don’t quite carry over because the actual "source-code" is not human interpretable.

Post reply on HN