Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Meta AI releases Code Llama 70B

twitter.com

Post reply on HN
Print view View original

71–80 of 311 posts

Re: Meta AI releases Code Llama 70B

#71

Post by 2OEH8eoCRo0 »

coder543 wrote: post #67
2OEH8eoCRo0 wrote: post #62

Earlier quoted context omitted.

How? It's larger than 64GB.

Quantization is highly effective at reducing memory and storage requirements, and it barely has any impact on quality (unless you take it to the extreme). Approximately no one should ever be running the full fat fp16 models during inference of any of these LLMs. That would be incredibly inefficient. I run 33B parameter models on my RTX 3090 (24GB VRAM) no problem. 70B should easily fit into 64GB of RAM.

I'm aware but is it still LLaMA 70B at that point?

Re: Meta AI releases Code Llama 70B

#72

Post by [deleted] »

ignoramous wrote: post #58
turnsout wrote: post #2

Given how good some of the smaller code models are (such as Deepseek Coder at 6.7B), I'll be curious to see what this 70B model is capable of!

AlphaCodium is the newest kid on the block that's SoTA pass@5 on coding tasks (authors claim at least 2x better than GPT4): https://github.com/Codium-ai/AlphaCodium As for small models, Microsoft has been making noise with the unreleased WaveCoder-Ultra-6.7b ( https://arxiv.org/abs/2312.14187 ).

[deleted]

Re: Meta AI releases Code Llama 70B

#73

Post by pyinstallwoes »

bk146 wrote: post #70

Can someone explain Meta's strategy with the open source models here? Genuine question, I don't fully undestand. (Please don't say "commoditize your complement" without explaining what exactly they're commoditizing...)

To be crowned the harbinger of AGI.

Re: Meta AI releases Code Llama 70B

#74

Post by apples_oranges »

bk146 wrote: post #70

Can someone explain Meta's strategy with the open source models here? Genuine question, I don't fully undestand. (Please don't say "commoditize your complement" without explaining what exactly they're commoditizing...)

OT: You „don‘t“ or you „don’t fully“ understand? ;)

(I try to train myself to say it right ..)

Re: Meta AI releases Code Llama 70B

#75

Post by eurekin »

bk146 wrote: post #70

Can someone explain Meta's strategy with the open source models here? Genuine question, I don't fully undestand. (Please don't say "commoditize your complement" without explaining what exactly they're commoditizing...)

Total speculation: Yann LeCun is there and he is really passionate about the technology and openness

Re: Meta AI releases Code Llama 70B

#76

Post by megaman821 »

austinpena wrote: post #5
colesantiago wrote: post #3

Llama is getting better and better, I heard this and Llama 3 will start to be good as GPT-4. Who would have thought that Meta, that has been chucking billions on the metaverse is on the forefront of Open Source AI. Not to mention their stock is up and they are worth $1TN, again. Not sure how I feel about this given the fact of all the scandals that have plagued them and the massive 1BN fine from the EU, Cambridge Ana…

I'm really curious what their goal is

They were going to make most this anyway for Instagram filters, chat stickers, internal coding tools, VR world generation, content moderation, etc. Might as well do a little bit extra work to open source it since it doesn't really compete with anything Meta is selling.

Re: Meta AI releases Code Llama 70B

#77

Post by rgbrgb »

2OEH8eoCRo0 wrote: post #62
moyix wrote: post #59

Earlier quoted context omitted.

You can run it on a Macbook M1/M2 with 64GB of RAM.

How? It's larger than 64GB.

Quantization can take it under 30GB (with quality degradation).

For example, take a look at the GGUF file sizes here: https://huggingface.co/TheBloke/Llama-2-70B-GGUF

Re: Meta AI releases Code Llama 70B

#78

Post by simonw »

bk146 wrote: post #70

Can someone explain Meta's strategy with the open source models here? Genuine question, I don't fully undestand. (Please don't say "commoditize your complement" without explaining what exactly they're commoditizing...)

AI seems like the Next Big Thing. Meta have put themselves at the center of the most exciting growth area in technology by releasing models they have trained.

They've gained an incredible amount of influence and mindshare.

Re: Meta AI releases Code Llama 70B

#80

Post by bryan_w »

bk146 wrote: post #70

Can someone explain Meta's strategy with the open source models here? Genuine question, I don't fully undestand. (Please don't say "commoditize your complement" without explaining what exactly they're commoditizing...)

Part of it is that they already had this developed for years (see alt text on uploaded images for example), and they want to ensure that new regulations don't hamper any of their future plans.

It costs them nothing to open it up, so why not. Kinda like all the rest of their GitHub repos.

Post reply on HN