Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

LLaMA: A foundational, 65B-parameter large language model

ai.facebook.com

Post reply on HN
Print view View original

71–80 of 209 posts

Re: LLaMA: A foundational, 65B-parameter large language model

#71

Post by skybrian »

fwlr wrote: post #10
mjburgess wrote: post #3

Earlier quoted context omitted.

> We release all our models to the research community. This is yet more evidence for the "AI isn't a competitive advantage" thesis. State-of-the-Art is a public resource, so competing with AI offers no "moat".

In terms of medieval warfare, what Facebook is doing here looks like filling the moat with rocks and dirt. OpenAI is worth billions, Microsoft is spending billions to retrofit most of their big offerings with AI, Google is doubtless also spending billions to integrate AI in their products. Their moat in all cases is “a blob of X billion weights trained on Y trillion tokens”. Facebook here is spending mere _millions_…

Not everyone. You have to apply for access.

Re: LLaMA: A foundational, 65B-parameter large language model

#72

Post by naasking »

vineyardmike wrote: post #53
recuter wrote: post #11

Earlier quoted context omitted.

I don't see such grand stratagems as being a likely explanation. It seems more likely that a bunch of dorks are running around unsupervised trying their best to make lemonade with whatever budgets they are given as nobody can realistically manage AI researches. Or at least, it seems that much like in political governance, events are suddenly outpacing the ability of corporate to react. In the case of OpenAI it is a "…

I think there is definitely a component of competition here. Facebook has no real way to monetize this (eg they won’t release an api a la OpenAI and they don’t own a search engine). Since they can’t monetize it… why not provide a bit of kindling to lower the barrier for everyone else to compete with your competitors. This strategy is called “commodize your complement”. If Facebook makes it easier to develop a google…

Facebook maybe can't make money from it, but they arguably could save money from it, for instance in automating some fact checking and moderation activities that they currently spend quite a bit of money on.

Re: LLaMA: A foundational, 65B-parameter large language model

#73

Post by freezed88 »

(creator of gpt index / llamaindex here https://github.com/jerryjliu/gpt_index)

Funny that we had just rebranded our tool from GPT Index to LlamaIndex about a week ago to avoid potential trademark issues with OpenAI, and turns out Meta has similar ideas around LLM+llama puns :). Must mean the name is good though!

Also very excited to try plugging in the LLaMa model into LlamaIndex, will report the results.

Re: LLaMA: A foundational, 65B-parameter large language model

#74

Post by simonw »

zozbot234 wrote: post #70
stevenhuang wrote: post #65

Earlier quoted context omitted.

How about alignment/ability to answer prompt queries and chain of thought reasoning capabilities? Without the fine tuning RLHF phase to make it like instructgpt I'm assuming it won't be as good as ChatGPT, is that right? How hard would it be to fine tune the 65B model on commodity hardware? Found answer here: > Out-of-scope use cases LLaMA is a base, or foundational, model. As such, it should not be used on downstrea…

Yes but fine tuning for RL is not expected to be hard. You're essentially limited by how much human feedback is available, so it's very different from training the foundational model on random bulk data.

At this point I fully expect that someone will release a usable RLHF-fine-tuned language model that can run on consumer hardware, based on the methodology used for LLaMA (and other similar papers e.g. https://github.com/FMInference/FlexGen ), at some point in the next 6-24 months.

Re: LLaMA: A foundational, 65B-parameter large language model

#75

Post by stevenhuang »

zozbot234 wrote: post #70
stevenhuang wrote: post #65

Earlier quoted context omitted.

How about alignment/ability to answer prompt queries and chain of thought reasoning capabilities? Without the fine tuning RLHF phase to make it like instructgpt I'm assuming it won't be as good as ChatGPT, is that right? How hard would it be to fine tune the 65B model on commodity hardware? Found answer here: > Out-of-scope use cases LLaMA is a base, or foundational, model. As such, it should not be used on downstrea…

Yes but fine tuning for RL is not expected to be hard. You're essentially limited by how much human feedback is available, so it's very different from training the foundational model on random bulk data.

Of course, but even still the level of scale, clean data, and human supervision needed may be significant. It's reported OpenAI used an army of humans to generate question answer prompts and rate the model output.

They kept the details closely guarded and only hinted at how they did RLHF and transitioned the architecture to self supervised learning.

Re: LLaMA: A foundational, 65B-parameter large language model

#76

Post by rcpt »

recuter wrote: post #34
mliker wrote: post #30

Earlier quoted context omitted.

this is a pretty biased and uninformed opinion. Pretty condescending to call it a "bunch of dorks...running around unsupervised."

I don't know what my bias is supposed to be but I called them dorks affectionately for one. The other replies are literally arguing that they are very extremely supervised where as I am speculating they are just eager to share their work for the right reasons and the eye of Sauron has yet to turn upon them. Inside knowledge I never claimed. Anything else I can help you with today? :)

> I called them dorks affectionately

Never in my life have I seen that word used affectionately

Re: LLaMA: A foundational, 65B-parameter large language model

#80

Post by CuriouslyC »

blablablub wrote: post #60

So what hardware do we need to run this model?

7 billion can run on 16+ gb GPUs as fp16, 14 billion can be run on 16+ gb if quantized to int8. 14G @ fp16 and 30G at int8 will require one of the 48 gb cards (less, but hardware mostly goes 24 -> 48).
Post reply on HN