Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Meta AI releases Code Llama 70B

twitter.com

Post reply on HN
Print view View original

261–270 of 311 posts

Re: Meta AI releases Code Llama 70B

#261

Post by hnfong »

mvkel wrote: post #257
chrishare wrote: post #192

Credit where credit is due, Meta has had a fantastic commitment towards open source ML. You love to see it.

Wasn't LLaMa originally a leak that they were then forced to spin into an open source contribution? Not to diminish the value of the contribution, but "commitment" is an interesting word choice.

It wasn't a leak in the typical sense. They sent the weights to pretty much everyone who asked nicely.

When you send something interesting to thousands of people without vetting their credentials, you'd expect the stuff to get "leaked" out eventually (and sooner rather than later).

I'd say it's more appropriate to say the weights were "pirated" than "leaked".

That said, you're probably correct that the community that quickly formed around the "pirated" weights might have influenced Zuckerberg to decide to make llama2's more freely accessible.

Re: Meta AI releases Code Llama 70B

#262

Post by eurekin »

famouswaffles wrote: post #148
eurekin wrote: post #120

Earlier quoted context omitted.

Given how megacorps spend millions on a whim (Disney with all recent flops) or, when just a single person wants it (Ms Flight Simulator?) - I wouldn't be surprised to be honest... But sure, sounds more reasonable

Disney didn't spend millions on a whim. It's just the reality of box office that even millions in investment are no guarantee for returns.

Financially, they have underperformed significantly over longer period of time (10 years):

  For shareholders, this subpar performance has destroyed value. Disney stock has underperformed the stocks
  of Disney’s self-selected proxy peers and the broader market over every relevant period during the last
  decade and during the tenure of each non-management director. Furthermore, it has underperformed since
  Bob Iger was first appointed CEO in 2005 – a period during which he has served as CEO or Executive
  Chairman (directing the Company’s creative endeavors in this role) for all but 11 months. Disney shareholders
  were once over $200 billion wealthier than they are now
Which is radically different from previous 90 years

https://trianpartners.com/wp-content/uploads/2023/12/Trian-N...

Re: Meta AI releases Code Llama 70B

#263

Post by dan_can_code »

madeofpalk wrote: post #251
dan_can_code wrote: post #225

Earlier quoted context omitted.

It's great. It's super easy to install ollama locally, `ollama run `, change the continue config to point to it, and it just works. It even has an offline option by disabling telemetry.

> Windows coming soon ugh, not so easy.

Yes, that is certainly a downside I forgot to mention. Sorry to get your hopes up.

Re: Meta AI releases Code Llama 70B

#264

Post by a_wild_dandan »

anonymousDan wrote: post #208

Can anyone tell me what kind of hardware setup would be needed to fine-tune something like this? Would you need a cluster of GPUs? What kind of size + GPU spec would you think is reasonable (e.g. wrt VRAM per GPU etc).

I just use my M2 MacBook Pro. Works great on big models.

Re: Meta AI releases Code Llama 70B

#265

Post by annjose »

israrkhan wrote: post #203
simonw wrote: post #92

Here's the model on Hugging Face: https://huggingface.co/codellama/CodeLlama-70b-hf

I hope someone will soon post a quantized version that I can run on my macbook pro.

Ollama has released the quantized version.

https://ollama.ai/library/codellama:70b https://x.com/ollama/status/1752034686615048367?s=20

Just need to run `ollama run codellama:70b` - pretty fast on macbook.

Re: Meta AI releases Code Llama 70B

#266

Post by raxxorraxor »

LVB wrote: post #163

I'm not very plugged into how to use these models, but I do love and pay for both ChatGPT and GitHub Copilot. How does one take a model like this (or a smaller version) and leverage it in VS Code? There's a dizzying array of GPT wrapper extensions for VS Code, many of which either seem like kind of junk (10 d/ls, no updates in a year), or just lead to another paid plan, at which point I might as well just keep my GH…

I use the plugin Twinny in conjunction with ollama to host the models. Easy setup and quite powerful assistance. You need a decent rig though, since you don't want any latency for features like autocomplete.

But even if you don't have a faster rig, you can still leverage it for slower tasks to generate docs or tests.

Twinny should really be more popular, didn't find a more powerful no-bullshit plugin for VSCode.

Re: Meta AI releases Code Llama 70B

#267

Post by raxxorraxor »

hnfong wrote: post #261
mvkel wrote: post #257

Earlier quoted context omitted.

Wasn't LLaMa originally a leak that they were then forced to spin into an open source contribution? Not to diminish the value of the contribution, but "commitment" is an interesting word choice.

It wasn't a leak in the typical sense. They sent the weights to pretty much everyone who asked nicely. When you send something interesting to thousands of people without vetting their credentials, you'd expect the stuff to get "leaked" out eventually (and sooner rather than later). I'd say it's more appropriate to say the weights were "pirated" than "leaked". That said, you're probably correct that the community that…

If the image synthesis is an example, I believe this to be a winning strategy in the long run to get the most competent model. The LLM space just moves a bit slower since the entry requirements for compute power are that much higher and data preparation is maybe a bit more "dry" than tagging images.

Re: Meta AI releases Code Llama 70B

#268

Post by fennecfoxy »

pandominium wrote: post #228

Everyone is mentioning using 4090 and a smaller model, but I rarely see an analysis where the energy consumption is used. I think Copilot is already highly subsidized by Microsoft. Let's say you use Copilot around 30% of your daily work hours. How much kWh does an opensource 7B or 13B model use then in a month on one 4090? EDIT: I think for a 13B at 30% use per day it comes around 30$/no on energy bill. So probably w…

Don't even need to do hard math: compare using co-pilot style LLM (bursts of 100% GPU every wee while) vs gaming on your 4090 (running at 100% for x hours).

Re: Meta AI releases Code Llama 70B

#270

Post by satvikpendem »

esafak wrote: post #158
gen220 wrote: post #84

Earlier quoted context omitted.

They're commoditizing the ability to generate viral content, which is the carrot that keeps peoples' eyeballs on the hedonic treadmill. More eyeball-time = more ad placements = more money. On the advertiser side, they're commoditizing the ability for companies to write more persuasively-targeted ads. Higher click-through rates = more money. [edit]: For models that generate code instead of content (TFA), it's obviousl…

That stuff ultimately drives people away. Who thinks "I need my daily fix of genAI memes, let me head to Facebook!"?

People on HN are not representative of the average Facebook user.
Post reply on HN