Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Meta wants its open source AI model to be as capable as OpenAI’s best model

wsj.com

Post reply on HN
Print view View original

31–40 of 62 posts

Re: Meta wants its open source AI model to be as capable as OpenAI’s best model

#31

Post by [deleted] »

_boffin_ wrote: post #28
kkielhofner wrote: post #22

Earlier quoted context omitted.

It’s the early web all over again. Sun, DEC, Microsoft, etc just assumed they would use their cash and market presence to dominate the web up, down, left, and right. LAMP showed up and ate their lunch. Today open source is the underpinnings for basically every startup over the past 25 years and everything from Android to MacOS to every browser rendering engine. An exclusionary list would be easier. A 2008 study[0] fr…

> Personally I don’t understand how anyone could think OpenAI, etc vs FOSS is going to be special or unique against what we know to be obvious. APIs... it's all about APIs and ease of use. Companies have data centers or could have data centers if they wanted, but a lot use AWS, MSFT, GCP, and friends.

[deleted]

Re: Meta wants its open source AI model to be as capable as OpenAI’s best model

#32

Post by AlexanderTheGr8 »

sdenton4 wrote: post #7

As the Socrates said, one must commoditize the complement.

I have always found concepts such as "commoditize the complement" very interesting. For some reason, they seem like pseudo explanations of behaviours we see from companies.

I am not entirely convinced that this was Meta's reasoning for releasing Llama. Like, I think Zuck is not thinking along these lines while open srcing LLMs.

How is Llama a complement of Meta's products? Meta wants more social interaction. It's a big stretch to assume LLMs will lead to more shareable social content (unless meta wants AI bots, in which case they don't need LLMs; they can just relax their moderation policies).

I honestly still don't have any good explanations for releasing Llama. Even the explanation of Meta gaining value from public improvements to Llama seems too vague and a stretch. Meta has enough engineers to make the most useful improvements themselves. The cost and effort of open srcing Llama is >> the value Meta gains from public improvements of Llama.

So, yeah this is still an open ques for me...

Re: Meta wants its open source AI model to be as capable as OpenAI’s best model

#33

Post by AlexanderTheGr8 »

Does anyone know why Meta is open srcing Llama? The 2 explanations I have heard are "commoditize the complement" and "take advantage of public improvements of Llama".

I have always found concepts such as "commoditize the complement" very interesting. For some reason, they seem like pseudo explanations of behaviours we see from companies. I am not entirely convinced that this was Meta's reasoning for releasing Llama. Like, I think Zuck is not thinking along these lines while open srcing LLMs.

How is Llama a complement of Meta's products? Meta wants more social interaction. It's a big stretch to assume LLMs will lead to more shareable social content (unless meta wants AI bots, in which case they don't need LLMs; they can just relax their moderation policies).

I honestly still don't have any good explanations for releasing Llama. Even the explanation of Meta gaining value from public improvements to Llama seems too vague and a stretch. Meta has enough engineers to make the most useful improvements themselves. The cost and effort of open srcing Llama is >> the value Meta gains from public improvements of Llama.

So, yeah this is still an open ques for me...

Re: Meta wants its open source AI model to be as capable as OpenAI’s best model

#34

Post by fragmede »

AlexanderTheGr8 wrote: post #33

Does anyone know why Meta is open srcing Llama? The 2 explanations I have heard are "commoditize the complement" and "take advantage of public improvements of Llama". I have always found concepts such as "commoditize the complement" very interesting. For some reason, they seem like pseudo explanations of behaviours we see from companies. I am not entirely convinced that this was Meta's reasoning for releasing Llama.…

One possibility is that they'll launch a hosted version of Llama-4 whenever they come out with that, and think the branding from Llama will help them sell that future version.

Re: Meta wants its open source AI model to be as capable as OpenAI’s best model

#35

Post by Merik »

placesalt wrote: post #15
sdenton4 wrote: post #7

As the Socrates said, one must commoditize the complement.

I think that was Churchill, but the point stands

https://www.joelonsoftware.com/2002/06/12/strategy-letter-v/

I’m guessing y’all are both being facetious :)

Re: Meta wants its open source AI model to be as capable as OpenAI’s best model

#36

Post by smoldesu »

consumer451 wrote: post #25
natch wrote: post #21

Earlier quoted context omitted.

I for one agree we should not, but also we shouldn't underestimate our opponents and assume that we are or will stay ahead.

I agree with that. But when ChatGPT was made public, I read of Chinese citizens complaining about how far behind they were. I see OSS LLMs along the same lines as the Rolls Royce Nene being sold to the USSR in 1946, or the captured AIM-9 being "the Rosetta Stone" for Soviet engineers as far as missile design. I mean, will Kremlin or CCP engineers contribute their PRs to OSS LLMs? This seems so utterly naive to me on…

> I mean, will Kremlin or CCP engineers contribute their PRs to OSS LLMs?

Nobody can. That's not how model training works. They can however finetune those models and redistribute them, and many do. It's trivial, and I've seen people from all over the world publish models on HuggingFace.

Will state-sponsored engineers upstream their work? No, but fat chance anyways. It's not like andrew Tannenbaum is crying crocodile tears over all the MINIX patches the NSA never gave back. Open Source work acknowledges these use cases, and it relies on the piety of law to do what is right. Countries like China and Russia don't regularly adhere to copyright, so there's no reason to highlight their failure to work with copyleft. It's a nothingburger.

> I see OSS LLMs along the same lines as the Rolls Royce Nene being sold to the USSR in 1946

Why? It's text. It's trained on a big pile of data you can go download and take to China in a plane on a flash drive: https://pile.eleuther.ai/

It's like exporting the IP equivalent of Silly Putty.

> I was once religious about OSS, but then I realized that not all products are the same.

Products? No. Software? Yes.

Re: Meta wants its open source AI model to be as capable as OpenAI’s best model

#37

Post by wrsh07 »

AlexanderTheGr8 wrote: post #33

Does anyone know why Meta is open srcing Llama? The 2 explanations I have heard are "commoditize the complement" and "take advantage of public improvements of Llama". I have always found concepts such as "commoditize the complement" very interesting. For some reason, they seem like pseudo explanations of behaviours we see from companies. I am not entirely convinced that this was Meta's reasoning for releasing Llama.…

I strongly endorse the commodify complement hypothesis, but here are others:

1. The only one that matters - the controlling founder wants to

2. It's great marketing. Make Facebook engineering (which is talented) seem cool. Also let their talented engineering teams flex their muscles a bit.

3. Investing in GPUs is a decent hedge in case there is something world changing coming (imagine Instagram reels but 50% of the content is ai generated. Or maybe just all ads are). Remember mobile - a previous paradigm shift - was an existential threat for Facebook

4. This is inline with the complement hypothesis, but what are you going to do with text you generate with llama? Some of that content will go right back on Facebook or one of their many platforms

5. WhatsApp business is a thing, and Facebook is talented enough at engineering that they can mimic any cool chat bot tools people build and integrate them into that

6. It's free for them (what are the downsides?) and reputationally gives them a certain kind of moral high ground compared to Google or OpenAI

7. Oh, the publicity will sell users on accessing the llama API hosted on Google (which pays Facebook money)

What are the downsides? Like seriously!

Re: Meta wants its open source AI model to be as capable as OpenAI’s best model

#39

Post by cendyne »

ComputerGuru wrote: post #17
superkuh wrote: post #2

In terms of making something that could beat a turing test the 65B/70B llama1/2 already are better than openai's currently offered models like gpt3.5-turbo or even the currently available output from gpt3.5 text-davinci-003. They've been so heavily "aligned" that they're insufferable and "As a large language model," everything even given an extensive pre-prompt in text completion mode. It wasn't always this way. Back…

What do you mean by “aligned?”

Alignment of large language models and machine learning is about shaping its behaviors to support the values of its maintainers and discourage behaviors that go against the same values.

Like, you can't get a positive response by asking it to write a phishing letter for you, and it will refuse to divulge information it might know which is considered personally identifiable information.

Unfortunately, the alignment applied since ChatGPTs launch has reduced its efficacy in other ways. While it behaves better, it may be less helpful or less accurate in the older models like 3, and 3.5 with respect to their debut.

Re: Meta wants its open source AI model to be as capable as OpenAI’s best model

#40

Post by ComputerGuru »

cendyne wrote: post #39
ComputerGuru wrote: post #17

Earlier quoted context omitted.

What do you mean by “aligned?”

Alignment of large language models and machine learning is about shaping its behaviors to support the values of its maintainers and discourage behaviors that go against the same values. Like, you can't get a positive response by asking it to write a phishing letter for you, and it will refuse to divulge information it might know which is considered personally identifiable information. Unfortunately, the alignment app…

Thanks for taking the time to answer!
Post reply on HN