Live data from Hacker News

Meta AI releases Code Llama 70B

twitter.com

31–40 of 311 posts

Re: Meta AI releases Code Llama 70B

#31
There is a bait and switch going on, and sam altman or mark zuckerberg are the first to tell you.

“No one can compete with us, but it’s cute to try! Make applications though” —almost direct quote from Sam Altman.

I have 64gb and an RTX 3090 and a macbook M3, and I already can’t run a lot of the newest models even in their quantized form.

The business model requires this to be a subscription service. At least as of today…

Re: Meta AI releases Code Llama 70B

#32

Llama is getting better and better, I heard this and Llama 3 will start to be good as GPT-4. Who would have thought that Meta, that has been chucking billions on the metaverse is on the forefront of Open Source AI. Not to mention their stock is up and they are worth $1TN, again. Not sure how I feel about this given the fact of all the scandals that have plagued them and the massive 1BN fine from the EU, Cambridge Ana…

I'm really curious what their goal is

I imagine their goal is to simultaneously show that Meta is still SotA when it comes to AI and at the same time feed a community of people who will work for free to essentially undermine OpenAI's competitive advantage and make life worse for Google since at the very least LLMs tend to be a better search engine for most topics.

There's far more risk if Meta were to try to directly compete with OpenAI and Microsoft on this. They'd have to manage the infra, work to acquire customers, etc, etc on top of building these massive models. If it's not a space they really want to be in, it's a space they can easily disrupt.

Meta's late game realization was that Google owned the web via search and Apple took over a lot of the mobile space with their walled garden. I suspect Meta's view now is that it's much easier to just prevent something like this from happening with AI early on.

Re: Meta AI releases Code Llama 70B

#34
post #18

Earlier quoted context omitted.

I'm really curious what their goal is

Commoditizing your complement. If all your competitors need a key technology to get ahead, you make a cheap/free version of it so that they can't use it as a competitive advantage.

The complement being the metaverse. You can’t handcraft the metaverse because it would be infeasible. If LLMs are a commodity that everyone has access to, then it can be done on the cheap.

Put another way - if OpenAI were the only game in town how much would they be charging for their product? They’re competing on price because competitors exist. Now imagine the price if a hypothetical high quality open source model existed that can customers can use for “free”.

That’s the future Meta wants. They weren’t getting rich selling shovels like cloud providers are, they want everyone digging. And everyone digs when the shovels are free.

Re: Meta AI releases Code Llama 70B

#35
post #31

There is a bait and switch going on, and sam altman or mark zuckerberg are the first to tell you. “No one can compete with us, but it’s cute to try! Make applications though” —almost direct quote from Sam Altman. I have 64gb and an RTX 3090 and a macbook M3, and I already can’t run a lot of the newest models even in their quantized form. The business model requires this to be a subscription service. At least as of to…

Realistically, what hardware would be required to run this? I assumed a RTX 3090 would be enough?

Re: Meta AI releases Code Llama 70B

#36
post #31

There is a bait and switch going on, and sam altman or mark zuckerberg are the first to tell you. “No one can compete with us, but it’s cute to try! Make applications though” —almost direct quote from Sam Altman. I have 64gb and an RTX 3090 and a macbook M3, and I already can’t run a lot of the newest models even in their quantized form. The business model requires this to be a subscription service. At least as of to…

Realistically, what hardware would be required to run this? I assumed a RTX 3090 would be enough?

A Studio Mac with an M1 Ultra (about 2800 USD used) is actually a really cost effective way to run in. Its total system power consumption is really low, even spitting out tokens at full tilt (<250W).

Re: Meta AI releases Code Llama 70B

#38
post #31

There is a bait and switch going on, and sam altman or mark zuckerberg are the first to tell you. “No one can compete with us, but it’s cute to try! Make applications though” —almost direct quote from Sam Altman. I have 64gb and an RTX 3090 and a macbook M3, and I already can’t run a lot of the newest models even in their quantized form. The business model requires this to be a subscription service. At least as of to…

A 70B model is quite accessible; just rent a data center GPU hourly. There are easy deployment services that are getting better all the time. Smaller models can be derived from the big ones to run on a MacBook running Apple Silicon. While the compute won’t be a match for Nvidia hardware, a MacBook can pack 128GB of RAM and run enormous models - albeit slowly.

Re: Meta AI releases Code Llama 70B

#39
post #23

Not sure who this is aimed at? The avg programmer probably doesn’t have the gear on hand to run this at the required pace Cool nonetheless

This is targeted towards GPU rental services like RunPod as well as API providers such as together AI. Together.ai is charging $0.90/1M tokens at 70B parameters. https://www.together.ai/pricing

Re: Meta AI releases Code Llama 70B

#40
post #23

Not sure who this is aimed at? The avg programmer probably doesn’t have the gear on hand to run this at the required pace Cool nonetheless

Yeah, but if your company wants to rent an H100, you can deploy this for your developers for much less than the cost of a developer…
Post reply on HN