Live data from Hacker News

Meta AI releases Code Llama 70B

twitter.com

21–30 of 311 posts

Re: Meta AI releases Code Llama 70B

#21

Llama is getting better and better, I heard this and Llama 3 will start to be good as GPT-4. Who would have thought that Meta, that has been chucking billions on the metaverse is on the forefront of Open Source AI. Not to mention their stock is up and they are worth $1TN, again. Not sure how I feel about this given the fact of all the scandals that have plagued them and the massive 1BN fine from the EU, Cambridge Ana…

I'm really curious what their goal is

To undermine the momentum of OpenAI.

If Meta were at the forefront, these models would not be openly available.

They are scrambling.

Re: Meta AI releases Code Llama 70B

#25

Llama is getting better and better, I heard this and Llama 3 will start to be good as GPT-4. Who would have thought that Meta, that has been chucking billions on the metaverse is on the forefront of Open Source AI. Not to mention their stock is up and they are worth $1TN, again. Not sure how I feel about this given the fact of all the scandals that have plagued them and the massive 1BN fine from the EU, Cambridge Ana…

with pytorch (& so many open publications), Meta has had a unimaginably strong contribution to ai for a while

Re: Meta AI releases Code Llama 70B

#26

Llama is getting better and better, I heard this and Llama 3 will start to be good as GPT-4. Who would have thought that Meta, that has been chucking billions on the metaverse is on the forefront of Open Source AI. Not to mention their stock is up and they are worth $1TN, again. Not sure how I feel about this given the fact of all the scandals that have plagued them and the massive 1BN fine from the EU, Cambridge Ana…

Model available, not open source. These models aren't open source because we don't have access to the data sets, nor the full code to train them, so we can't recreate the models even if we had the GPU time available to recreate them.

Re: Meta AI releases Code Llama 70B

#27

Llama is getting better and better, I heard this and Llama 3 will start to be good as GPT-4. Who would have thought that Meta, that has been chucking billions on the metaverse is on the forefront of Open Source AI. Not to mention their stock is up and they are worth $1TN, again. Not sure how I feel about this given the fact of all the scandals that have plagued them and the massive 1BN fine from the EU, Cambridge Ana…

I'm really curious what their goal is

If you go by what Zuck says, he calls this out in previous earnings reports and interviews[1]. It mainly boils down to 2 things:

1. Similar to other initiatives (mainly opencompute but also PyTorch, React etc), community improvements help them improve their own infra and helps attract talent.

2. Helping people create better content ultimately improves quality of content on their platforms (Both FoA & RL)

Sources:

[1]Interview with verge: https://www.theverge.com/23889057/mark-zuckerberg-meta-ai-el... . Search for "regulatory capture right now with AI"

> Zuck: ... And we believe that it’s generally positive to open-source a lot of our infrastructure for a few reasons. One is that we don’t have a cloud business, right? So it’s not like we’re selling access to the infrastructure, so giving it away is fine. And then, when we do give it away, we generally benefit from innovation from the ecosystem, and when other people adopt the stuff, it increases volume and drives down prices.

> Interviewer: Like PyTorch, for example?

> Zuck: When I was talking about driving down prices, I was thinking about stuff like Open Compute, where we open-sourced our server designs, and now the factories that are making those kinds of servers can generate way more of them because other companies like Amazon and others are ordering the same designs, that drives down the price for everyone, which is good.

Re: Meta AI releases Code Llama 70B

#28
post #2

Given how good some of the smaller code models are (such as Deepseek Coder at 6.7B), I'll be curious to see what this 70B model is capable of!

My personal experience is that Deepseek far exceeds code llama of the same size, but it was released quite a while ago.

Agreed—I hope Meta studied Deepseek's approach. The idea of a Deepseek Coder at 70B would be exciting.
Post reply on HN