Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Meta AI releases Code Llama 70B

twitter.com

Post reply on HN
Print view View original

41–50 of 311 posts

Re: Meta AI releases Code Llama 70B

#41

Post by YetAnotherNick »

b33j0r wrote: post #31

There is a bait and switch going on, and sam altman or mark zuckerberg are the first to tell you. “No one can compete with us, but it’s cute to try! Make applications though” —almost direct quote from Sam Altman. I have 64gb and an RTX 3090 and a macbook M3, and I already can’t run a lot of the newest models even in their quantized form. The business model requires this to be a subscription service. At least as of to…

> At least as of today…

This is the exact opposite of bait and switch. The current model couldn't be un-opensourced and over time it will just become easier to run it.

Also unless there is reason to believe that prompt engineering of different model families is very different(which honestly I don't believe), there is no effect of baiting. I believe it will always be the case that best 2-3 models would be closed weights.

Re: Meta AI releases Code Llama 70B

#42

Post by kkzz99 »

kuczmama wrote: post #35
b33j0r wrote: post #31

There is a bait and switch going on, and sam altman or mark zuckerberg are the first to tell you. “No one can compete with us, but it’s cute to try! Make applications though” —almost direct quote from Sam Altman. I have 64gb and an RTX 3090 and a macbook M3, and I already can’t run a lot of the newest models even in their quantized form. The business model requires this to be a subscription service. At least as of to…

Realistically, what hardware would be required to run this? I assumed a RTX 3090 would be enough?

RTX 3090 has 24GB of memory, a quantized llama70b takes around 60GB of memory. You can offload a few layers on the gpu, but most of them will run on the CPU with terrible speeds.

Re: Meta AI releases Code Llama 70B

#43

Post by keriati1 »

blackoil wrote: post #33
Havoc wrote: post #23

Not sure who this is aimed at? The avg programmer probably doesn’t have the gear on hand to run this at the required pace Cool nonetheless

How feasible would it be too fine tune using internal code and have an enterprise copilot.

We actually run already in-house ollama server prototype for coding assistance with deepseek coder and it is pretty good. Now if we would get a model for this, that is on chatgpt 4 level, I would be super happy.

Re: Meta AI releases Code Llama 70B

#44

Post by michaelt »

Arainach wrote: post #11
austinpena wrote: post #5

Earlier quoted context omitted.

I'm really curious what their goal is

Disclaimer: I do not work at Meta, but I work at a large tech company which competes with them. I don't work in AI, although if my VP asks don't tell them I said that or they might lay me off. Multiple of their major competitors/other large tech companies are trying to monetize LLMs. OpenAI maneuvering an early lead into a dominant position would be another potential major competitor. If releasing these models slows…

Why?

What benefit is there to grabbing market share from your competitors... in a business you don't even want to be in?

By that logic you could justify any bizarre business decision. Should Google launch a social network, to hurt their competitor Facebook? Should Facebook, Amazon and Microsoft each launch a phone?

Re: Meta AI releases Code Llama 70B

#45

Post by whimsicalism »

turnsout wrote: post #28
jasonjmcghee wrote: post #4

Earlier quoted context omitted.

My personal experience is that Deepseek far exceeds code llama of the same size, but it was released quite a while ago.

Agreed—I hope Meta studied Deepseek's approach. The idea of a Deepseek Coder at 70B would be exciting.

The “approach” is likely just training on more tokens.

Re: Meta AI releases Code Llama 70B

#46

Post by andy99 »

ramshanker wrote: post #14

Are these trained on internal Code bases or just the public repositories?

The github [0] hasn't been fully updated, but it links to a paper [1] that describes how the smaller code llama models were trained. It would be a good guess that this model is similar.

[0] https://github.com/facebookresearch/codellama [1] https://ai.meta.com/research/publications/code-llama-open-fo...

Re: Meta AI releases Code Llama 70B

#47

Post by whimsicalism »

b33j0r wrote: post #31

There is a bait and switch going on, and sam altman or mark zuckerberg are the first to tell you. “No one can compete with us, but it’s cute to try! Make applications though” —almost direct quote from Sam Altman. I have 64gb and an RTX 3090 and a macbook M3, and I already can’t run a lot of the newest models even in their quantized form. The business model requires this to be a subscription service. At least as of to…

the connection to the classic bait-and-switch seems tenuous at best

Re: Meta AI releases Code Llama 70B

#48

Post by moyix »

ttul wrote: post #38
b33j0r wrote: post #31

There is a bait and switch going on, and sam altman or mark zuckerberg are the first to tell you. “No one can compete with us, but it’s cute to try! Make applications though” —almost direct quote from Sam Altman. I have 64gb and an RTX 3090 and a macbook M3, and I already can’t run a lot of the newest models even in their quantized form. The business model requires this to be a subscription service. At least as of to…

A 70B model is quite accessible; just rent a data center GPU hourly. There are easy deployment services that are getting better all the time. Smaller models can be derived from the big ones to run on a MacBook running Apple Silicon. While the compute won’t be a match for Nvidia hardware, a MacBook can pack 128GB of RAM and run enormous models - albeit slowly.

My Macbook has a mere 64GB and that's plenty to run 70B models at 4-bit :) LM Studio is very nice for this.

Re: Meta AI releases Code Llama 70B

#49

Post by [deleted] »

ttul wrote: post #38
b33j0r wrote: post #31

There is a bait and switch going on, and sam altman or mark zuckerberg are the first to tell you. “No one can compete with us, but it’s cute to try! Make applications though” —almost direct quote from Sam Altman. I have 64gb and an RTX 3090 and a macbook M3, and I already can’t run a lot of the newest models even in their quantized form. The business model requires this to be a subscription service. At least as of to…

A 70B model is quite accessible; just rent a data center GPU hourly. There are easy deployment services that are getting better all the time. Smaller models can be derived from the big ones to run on a MacBook running Apple Silicon. While the compute won’t be a match for Nvidia hardware, a MacBook can pack 128GB of RAM and run enormous models - albeit slowly.

[deleted]

Re: Meta AI releases Code Llama 70B

#50

Post by blackoil »

austinpena wrote: post #5
colesantiago wrote: post #3

Llama is getting better and better, I heard this and Llama 3 will start to be good as GPT-4. Who would have thought that Meta, that has been chucking billions on the metaverse is on the forefront of Open Source AI. Not to mention their stock is up and they are worth $1TN, again. Not sure how I feel about this given the fact of all the scandals that have plagued them and the massive 1BN fine from the EU, Cambridge Ana…

I'm really curious what their goal is

Devil's advocate: they have to build it anyway for Meta verse and in general. Management has no interest in going into cloud business. They had Parse long time back but that is done. So why not to release it. They are getting goodwill/mindshare, may set up industry standard and get community benefit. It isn't very different from React, Torch etc.
Post reply on HN