I am not deep into llms so I ask this. From my understanding, their last model was open source but it was in a way that you can use them but the inner working were "hidden"/not transparent. With the new model, I am seeing alot of how open source they are and can be build upon. Is it now completely open source or similar to their last models ?
they give you the code and they give you the model it runs, and you can customise and redistribute both. It's all open source in that respect. What people are complaining about (totally unreasonably in my view) is obviously Meta is not "open sourcing" all the training data, so nobody can retrain the model from scratch themselves. This argument to me is just silly. The whole point of these models is they distil pretra…
Open source AI is the path forward
661–670 of 936 posts
Re: Open source AI is the path forward
#662On a more serious note, I don't really buy his arguments about safety. First, widespread AI does not reduce unintentional harm but increases it, because the rate of accident is compound. Second, the chance of success for threat actors will increase, because of the asymmetric advantage of gaining access to all open information and hiding their own information. But there is no reverse at this point, I enjoy it while it lasts, AGI will come sooner or later anyway.
Re: Open source AI is the path forward
#663Earlier quoted context omitted.
This isn't a dilemma at all. If Facebook can't release data it trains on because it would compromise user privacy, it is already a significant privacy violation that should be a scandal, and if it would prompt some regulatory or legislative remedies against Facebook for them to release the data, it should do the same for releasing the trained model, even through an API. The only reason people don't think about it thi…
> If Facebook can't release data it trains on because it would compromise user privacy, it is already a significant privacy violation that should be a scandal Thousands of entities would scramble to sue Facebook over any released dataset no matter what the privacy implications of the dataset are. It's just not worth it in any world. I believe you are not thinking of this problem from the view of the PM or VPs that wo…
The ire people have toward tech companies right now is, like most ire, perhaps in places overreaching. But it is mostly justified by the real actions of tech companies, and facebook has done more to deserve it than most. The thought process you just described sounds like an accurate prediction of the mindset and culture of a VP within Facebook, and I'd like you to reflect on it for a sec. Basically, you rightly point out that the org releasing what data they have would likely invite lawsuits, and then you proceeded to do some kind of insane offscreen mental gymnastics that allow this reality to mean nothing to you but that the unwashed masses irrationally hate the company for some unknowable reason
Like you're talking about a company that has spent the last decade buying competitors to maintain an insane amount of control over billions of users' access to their friends, feeding them an increasingly degraded and invasive channel of information that also from time to time runs nonconsensual social experiments on them, and following even people who didn't opt in around the internet through shady analytics plugins in order to sell dossiers of information on them to whoever will pay. What do you think it is? Are people just jealous of their success, or might they have some legit grievances that may cause them to distrust and maybe even loathe such an entity? It is hard for me to believe Facebook has a dataset large enough to train a current-gen LLM that wouldn't also feel, viscerally, to many, like a privacy violation. Whether any party that felt this way could actually win a lawsuit is questionable though, as the US doesn't really have signficant privacy laws, and this is partially due to extensive collaboration with, and lobbying by, Facebook and other tech companies who do mass-surveillance of this kind
I remember a movie called Das Leben der Anderen (2006) (Officially translated as "the lives of others") which got accolades for how it could make people who hadn't experienced it feel how unsettling the surveillance state of East Germany was, and now your average American is more comprehensively surveilled than the Stasi could have imagined, and this is in large part due to companies like facebook
Frankly, I'm not an AGI doomer, but if the capabilities of near-future AI systems are even in the vague ballpark of the (fairly unfounded) claims the American tech monopolies make about them, it would be an unprecedented disaster on a global scale if those companies got there first, so inasmuch as we view "AGI research" as something that's inevitably going to hit milestones in corporate labs with secretive datasets, I think we should absolutely kill it to whatever degree is possible, and that's as someone who truly, deeply believes that AI research has been beneficial to humanity and could continue to become moreso
Re: Open source AI is the path forward
#664Huge companies like facebook will often argue for solutions that on the surface, seem to be in the public interest. But I have strong doubts they (or any other company) actually believe what they are saying. Here is the reality: - Facebook is spending untold billions on GPU hardware. - Facebook is arguing in favor of open sourcing the models, that they spent billions of dollars to generate, for free...? It follows th…
The reason for Meta making their model open source is rather simple: They receive an unimaginable amount of free labor, and their license only excludes their major competitors to ensure mass adoption without benefiting their competition (Microsoft, Google, Alibaba, etc). Public interest, philanthropy, etc are just nice little marketing bonuses as far as they're concerned (otherwise they wouldn't be including this lic…
Re: Open source AI is the path forward
#665Earlier quoted context omitted.
AI might cannabalize a lot of first gen AI businesses.
What Meta is doing is borderline market distortion. It's not that they have figured out some magic sauce they are happy to share. They are just deciding to burn brute force money that they made elsewhere and give their stuff away below cost, first of all because they can.
Re: Open source AI is the path forward
#666Open source "AI" is a proxy for democratising and making (much) more widely useful the goodies of high performance computing (HPC). The HPC domain (data and compute intensive applications that typically need vector, parallel or other such architectures) have been around for the longest time, but confined to academic / government tasks. LLM's with their famous "matrix multiply" at their very core are basically demolis…
Re: Open source AI is the path forward
#667Sure but under what license? Because slapping “open source” on the model doesn’t make it open source if it’s not actually license that way. The 3.1 license still contains their non-commercial clause (over 700m users) and requires derivatives, whether fine tunings or trained on generated data, to use the llama name.
Llama could change the license on later versions to kill your business and you have no options as you don't know how they trained it or have the budget to.
It's not much more free than binary software.
Re: Open source AI is the path forward
#668Re: Open source AI is the path forward
#669Earlier quoted context omitted.
Neither of those usages relate to IT, they both are about sources of intelligence (espionage). Even if they were, the OSI definition won, nobody is using the definitions from 1995 CIA or the 1996 InfoConWar book in the realm of IT, not even Facebook. The community has the authority to complain about companies mis-labelling their pork products as vegan, even if nobody has a registered trademark on the term vegan. Woul…
Would you tell people to shut up about that case because they don't have a registered trademark? I especially like how I'm the one telling people to "shut up" all of a sudden. As for the rest, see my other reply.
Re: Open source AI is the path forward
#670I never thought I would say this but thanks Meta. *I reserve the right to remove this praise if they abuse this open source model position in the future.