Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Meta AI releases Code Llama 70B

twitter.com

Post reply on HN
Print view View original

111–120 of 311 posts

Re: Meta AI releases Code Llama 70B

#111

Post by IshKebab »

pchristensen wrote: post #86
bk146 wrote: post #70

Can someone explain Meta's strategy with the open source models here? Genuine question, I don't fully undestand. (Please don't say "commoditize your complement" without explaining what exactly they're commoditizing...)

Meta doesn't have an AI "product" competing with OpenAI, Google's Bard, etc. But they use AI extensively internally. This is roughly a byproduct of their internal AI work that they're already doing, and fostering open source AI development puts incredible pressure on the AI products and their owners. If Meta can help prevent there from being an AI monopoly company, but rather an ecosystem of comparable products, then…

Google releasing a (very popular) web browser gives them direct control of web standards. What does this give Facebook?

Re: Meta AI releases Code Llama 70B

#112

Post by coder543 »

andy99 wrote: post #105
2OEH8eoCRo0 wrote: post #71

Earlier quoted context omitted.

I'm aware but is it still LLaMA 70B at that point?

It's a legit question, the model will be worse in some way... I've seen it discussed that all things being equal more parameters is better (meaning it's better to take a big model and quantized it to fit in memory than use a smaller unquantized model that fits), but a quantized model wouldn't be expected to run identically to or as well as the full model.

You don’t stop being andy99 just because you’re a little tired, do you? Being tired makes everyone a little less capable at most things. Sometimes, a lot less capable.

In traditional software, the same program compiled for 32-bit and 64-bit architectures won’t be able to handle all of the same inputs, because the 32-bit version is limited by the available address space. It’s still the same program.

If we’re not willing to declare that you are a completely separate person when you’re tired, or that 32-bit and 64-bit versions are completely different programs, then I don’t think it’s worth getting overly philosophical about quantization. A quantized model is still the same model.

The quality loss from using 4+ bit quantization is minimal, in my experience.

Yes, it has a small impact on accuracy, but with massive efficiency gains. I don’t really think anyone should be running the full models outside of research in the first place. If anything, the quantized models should be considered the “real” models, and the full fp16/fp32 model should just be considered a research artifact distinct from the model. But this philosophical rabbit hole doesn’t seem to lead anywhere interesting to me.

Various papers have shown that 4-bit quantization is a great balance. One example: https://arxiv.org/pdf/2212.09720.pdf

Re: Meta AI releases Code Llama 70B

#113

Post by junon »

nemothekid wrote: post #99
3pt14159 wrote: post #87

Earlier quoted context omitted.

Something like meta.com/our_own_tech_handles_this

Not sure it's preferable to hire people at fb salaries to maintain a link shortener than just to use a reputable free one?

Every big company has one of these anyway, and usually more involved (internal DNS, VPN, etc). A link shortener is like an interview question.

Re: Meta AI releases Code Llama 70B

#114

Post by [deleted] »

Lerc wrote: post #98
bk146 wrote: post #70

Can someone explain Meta's strategy with the open source models here? Genuine question, I don't fully undestand. (Please don't say "commoditize your complement" without explaining what exactly they're commoditizing...)

If they hadn't opened the models the llama series would just be a few sub-GPT4 models. Opening the models has created a wealth of development that has built upon those models. Alone, it was unlikely they would become a major player in a field that might be massively important. With a large community building upon their base they have a chance to influence the direction of development and possibly prevent a proprietar…

[deleted]

Re: Meta AI releases Code Llama 70B

#115

Post by jejeyyy77 »

blackoil wrote: post #33
Havoc wrote: post #23

Not sure who this is aimed at? The avg programmer probably doesn’t have the gear on hand to run this at the required pace Cool nonetheless

How feasible would it be too fine tune using internal code and have an enterprise copilot.

already been done

Re: Meta AI releases Code Llama 70B

#116

Post by eganist »

IshKebab wrote: post #111
pchristensen wrote: post #86

Earlier quoted context omitted.

Meta doesn't have an AI "product" competing with OpenAI, Google's Bard, etc. But they use AI extensively internally. This is roughly a byproduct of their internal AI work that they're already doing, and fostering open source AI development puts incredible pressure on the AI products and their owners. If Meta can help prevent there from being an AI monopoly company, but rather an ecosystem of comparable products, then…

Google releasing a (very popular) web browser gives them direct control of web standards. What does this give Facebook?

OP already mentioned that it adds additional hurdles for possible future tech giants to have to cross on their quest.

It's akin to a Great Filter, if such an analogy helps. If Meta's open models make a company's closed models uneconomical for others to consume, then the business case for those models is compromised and the odds of them growing to a size where they can compete with Meta in other ways is mitigated a bit.

Re: Meta AI releases Code Llama 70B

#117

Post by eurekin »

keriati1 wrote: post #107
eurekin wrote: post #89

Earlier quoted context omitted.

Did you finetune a model?

No, we went with RAG pipeline approach as we assume things change too fast.

Thanks! Any details how you chunk and find the relevant code?

Or how you deal with context length? I.e. do you send anything other than the current file? How is the prompt constructed?

Re: Meta AI releases Code Llama 70B

#118

Post by Arainach »

michaelt wrote: post #44
Arainach wrote: post #11

Earlier quoted context omitted.

Disclaimer: I do not work at Meta, but I work at a large tech company which competes with them. I don't work in AI, although if my VP asks don't tell them I said that or they might lay me off. Multiple of their major competitors/other large tech companies are trying to monetize LLMs. OpenAI maneuvering an early lead into a dominant position would be another potential major competitor. If releasing these models slows…

Why? What benefit is there to grabbing market share from your competitors... in a business you don't even want to be in? By that logic you could justify any bizarre business decision. Should Google launch a social network, to hurt their competitor Facebook? Should Facebook, Amazon and Microsoft each launch a phone?

>Should Facebook, Amazon and Microsoft each launch a phone?

* https://www.lifewire.com/whatever-happened-to-the-facebook-p...

* https://en.wikipedia.org/wiki/Fire_Phone

* https://en.wikipedia.org/wiki/Windows_Phone

Re: Meta AI releases Code Llama 70B

#119

Post by theGnuMe »

bk146 wrote: post #70

Can someone explain Meta's strategy with the open source models here? Genuine question, I don't fully undestand. (Please don't say "commoditize your complement" without explaining what exactly they're commoditizing...)

Bill Gurly has a good perspective on it.

Essentially, you mitigate IP claims and reduce vendor dependency.

https://eightify.app/summary/technology-and-software/the-imp...

Re: Meta AI releases Code Llama 70B

#120

Post by eurekin »

skottenborg wrote: post #106
eurekin wrote: post #75

Earlier quoted context omitted.

Total speculation: Yann LeCun is there and he is really passionate about the technology and openness

I doubt personal passions would merit the company funding required for such big models.

Given how megacorps spend millions on a whim (Disney with all recent flops) or, when just a single person wants it (Ms Flight Simulator?) - I wouldn't be surprised to be honest...

But sure, sounds more reasonable

Post reply on HN