Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

LLaMA: A foundational, 65B-parameter large language model

ai.facebook.com

Post reply on HN
Print view View original

11–20 of 209 posts

Re: LLaMA: A foundational, 65B-parameter large language model

#11

Post by recuter »

fwlr wrote: post #10
mjburgess wrote: post #3

Earlier quoted context omitted.

> We release all our models to the research community. This is yet more evidence for the "AI isn't a competitive advantage" thesis. State-of-the-Art is a public resource, so competing with AI offers no "moat".

In terms of medieval warfare, what Facebook is doing here looks like filling the moat with rocks and dirt. OpenAI is worth billions, Microsoft is spending billions to retrofit most of their big offerings with AI, Google is doubtless also spending billions to integrate AI in their products. Their moat in all cases is “a blob of X billion weights trained on Y trillion tokens”. Facebook here is spending mere _millions_…

I don't see such grand stratagems as being a likely explanation. It seems more likely that a bunch of dorks are running around unsupervised trying their best to make lemonade with whatever budgets they are given as nobody can realistically manage AI researches. Or at least, it seems that much like in political governance, events are suddenly outpacing the ability of corporate to react.

In the case of OpenAI it is a "nudge humanity into a more wholesome direction" because a lot of them went on an acid fueled toxic affective altruism bender. And in this case it is "release all the things while Zuck is still obsessed with VR --for science!".

I like this other group better. But it is disturbing that it can stop at any moment. Probably why they are doing it, while they still can.

Re: LLaMA: A foundational, 65B-parameter large language model

#12

Post by PoignardAzur »

fwlr wrote: post #10
mjburgess wrote: post #3

Earlier quoted context omitted.

> We release all our models to the research community. This is yet more evidence for the "AI isn't a competitive advantage" thesis. State-of-the-Art is a public resource, so competing with AI offers no "moat".

In terms of medieval warfare, what Facebook is doing here looks like filling the moat with rocks and dirt. OpenAI is worth billions, Microsoft is spending billions to retrofit most of their big offerings with AI, Google is doubtless also spending billions to integrate AI in their products. Their moat in all cases is “a blob of X billion weights trained on Y trillion tokens”. Facebook here is spending mere _millions_…

Which kind of suggests Microsoft made a really bad move antagonizing the open-source community with Gtihub Copilot.

They got a few years of lead time in the "AI codes for you" market, but in exchange permanently soured a significant fraction of their potential userbase who will turn to open-source alternatives soon anyway.

I wonder if they'd have been better served focusing on selling Azure usage and released Copilot as an open-source product.

Re: LLaMA: A foundational, 65B-parameter large language model

#13

Post by jerpint »

ninjin wrote: post #9
rvz wrote: post #5

> To maintain integrity and prevent misuse, we are releasing our model under a noncommercial license focused on research use cases. Access to the model will be granted on a case-by-case basis to academic researchers; those affiliated with organizations in government, civil society, and academia; and industry research laboratories around the world. People interested in applying for access can find the link to the appl…

Facebook continues to appropriate the word “open” [1], which is sad really. There are plenty of good words they and others could use instead. [1]: https://news.ycombinator.com/item?id=32079558

I just signed some form with license to maybe possibly get access to the model… not very open indeed

Re: LLaMA: A foundational, 65B-parameter large language model

#14

Post by fwlr »

recuter wrote: post #11
fwlr wrote: post #10

Earlier quoted context omitted.

In terms of medieval warfare, what Facebook is doing here looks like filling the moat with rocks and dirt. OpenAI is worth billions, Microsoft is spending billions to retrofit most of their big offerings with AI, Google is doubtless also spending billions to integrate AI in their products. Their moat in all cases is “a blob of X billion weights trained on Y trillion tokens”. Facebook here is spending mere _millions_…

I don't see such grand stratagems as being a likely explanation. It seems more likely that a bunch of dorks are running around unsupervised trying their best to make lemonade with whatever budgets they are given as nobody can realistically manage AI researches. Or at least, it seems that much like in political governance, events are suddenly outpacing the ability of corporate to react. In the case of OpenAI it is a "…

The researchers and engineers and other assorted dorks who built it weren’t thinking about moats, for sure, I agree with you there. But I guarantee you that the metaphor of medieval warfare was on the minds of the executives and legal team deciding whether to let the team release their work in this way.

Re: LLaMA: A foundational, 65B-parameter large language model

#15

Post by swatcoder »

rvz wrote: post #5

> To maintain integrity and prevent misuse, we are releasing our model under a noncommercial license focused on research use cases. Access to the model will be granted on a case-by-case basis to academic researchers; those affiliated with organizations in government, civil society, and academia; and industry research laboratories around the world. People interested in applying for access can find the link to the appl…

> Even if you did, you can't use it for your commercial product anyway.

And of course, the irony is that its not commercial products that endanger “integrity and misuse”.

The looming misuse of LLM’s is cheap content flooding by spammers, black hats, and propaganda bots — and those people don’t care about licenses, and will inevitably defeat any watermarks meant to prevent or track leaks.

Re: LLaMA: A foundational, 65B-parameter large language model

#17

Post by freeqaz »

PoignardAzur wrote: post #12
fwlr wrote: post #10

Earlier quoted context omitted.

In terms of medieval warfare, what Facebook is doing here looks like filling the moat with rocks and dirt. OpenAI is worth billions, Microsoft is spending billions to retrofit most of their big offerings with AI, Google is doubtless also spending billions to integrate AI in their products. Their moat in all cases is “a blob of X billion weights trained on Y trillion tokens”. Facebook here is spending mere _millions_…

Which kind of suggests Microsoft made a really bad move antagonizing the open-source community with Gtihub Copilot. They got a few years of lead time in the "AI codes for you" market, but in exchange permanently soured a significant fraction of their potential userbase who will turn to open-source alternatives soon anyway. I wonder if they'd have been better served focusing on selling Azure usage and released Copilot…

How did Microsoft sour developers with Copilot? I know dozens of people that pay for it (including myself) and I feel like it is widely regarded as a "no brainer" for the price that it's offered at.

Please help me understand!

Re: LLaMA: A foundational, 65B-parameter large language model

#18

Post by swyx »

> To maintain integrity and prevent misuse, we are releasing our model under a noncommercial license focused on research use cases.

if say i wanted to replicate this paper for commercial use, what would it take and how do i get started? would FB have a basis for objection?

Re: LLaMA: A foundational, 65B-parameter large language model

#19

Post by sandkoan »

freeqaz wrote: post #17
PoignardAzur wrote: post #12

Earlier quoted context omitted.

Which kind of suggests Microsoft made a really bad move antagonizing the open-source community with Gtihub Copilot. They got a few years of lead time in the "AI codes for you" market, but in exchange permanently soured a significant fraction of their potential userbase who will turn to open-source alternatives soon anyway. I wonder if they'd have been better served focusing on selling Azure usage and released Copilot…

How did Microsoft sour developers with Copilot? I know dozens of people that pay for it (including myself) and I feel like it is widely regarded as a "no brainer" for the price that it's offered at. Please help me understand!

Presumably, because they trained Copilot on billions of lines of, often licensed, code (without permission), that Copilot has a tendency to regurgitate verbatim, without said license.

Re: LLaMA: A foundational, 65B-parameter large language model

#20

Post by minimaxir »

swyx wrote: post #18

> To maintain integrity and prevent misuse, we are releasing our model under a noncommercial license focused on research use cases. if say i wanted to replicate this paper for commercial use, what would it take and how do i get started? would FB have a basis for objection?

A metric-ton of GPU compute, for starters.
Post reply on HN