Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Code Llama, a state-of-the-art large language model for coding

ai.meta.com

Post reply on HN
Print view View original

171–180 of 525 posts

Re: Code Llama, a state-of-the-art large language model for coding

#171

Post by ilaksh »

jebarker wrote: post #131
ilaksh wrote: post #61

Earlier quoted context omitted.

Have you tried Code Llama? How do you know it can't do it already? In my applications, GPT-4 connected to a VM or SQL engine can and does debug code when given error messages. "Reliably" is very subjective. The main problem I have seen is that it can be stubborn about trying to use outdated APIs and it's not easy to give it a search result with the correct API. But with a good web search and up to date APIs, it can d…

What does "GPT-4 connected to a VM or SQL engine" mean?

https://aidev.codes shows connected to VM.

Re: Code Llama, a state-of-the-art large language model for coding

#172

Post by _joel »

ohyes wrote: post #121
ramesh31 wrote: post #24

Earlier quoted context omitted.

>Even the 7B model of code llama seems to be competitive with Codex, the model behind copilot It's extremely good. I keep a terminal tab open with 7b running for all of my "how do I do this random thing" questions while coding. It's pretty much replaced Google/SO for me.

What hardware do you have that lets you run 7b and do other stuff at the same time?

If you're willing to sacrifice token/s you can even run these on your phone.

Re: Code Llama, a state-of-the-art large language model for coding

#173

Post by criley2 »

dangerwill wrote: post #103

It's really sad how everyone here is fawning over tech that will destroy you own livelihoods. "AI won't take your job, those who use AI will" is purely short term, myopic thinking. These tools are not aimed to help workers, the end goal is to make it so you don't need to be an engineer to build software, just let the project manager or director describe the system they want and boom there it is. You can scream that t…

You can say this about every major invention. The loom destroyed jobs! The engine destroyed jobs! So on and so forth.

This view is critically flawed in two major ways:

1) AI is not anywhere near being able to replace the majority of what developers do on a product team. We are decades away from a PM at Facebook being able to type "make a twitter clone that uses instagram login and can scale to 1 billion" and have an AI just do it.

2) Programming and product work is not zero sum. The more we can do means the more product we can make. It means more products can be made overall. After the loom came out, we simply made more clothes than ever before and in the process created a ton of jobs. We are not at some peak software point where we've completely saturated all humanity's need for software or profitable software and thus tools that increase efficiency don't put us out of work.

And frankly, if we develop the kind of general AI that accept a query like "make a facebook competitor capable of scaling to 10 billion" and simply do it, inventing whatever languages, frameworks, hardware, processors, patterns, methodologies, global datacenters handling global politics and law, etc, etc necessary to accomplish such a task, then so be it. I welcome the overlords!

Re: Code Llama, a state-of-the-art large language model for coding

#174

Post by liuliu »

Draiken wrote: post #166

As a complete noob at actually running these models, what kind of hardware are we talking here? Couldn't pick that up from the README. I absolutely love the idea of using one of these models without having to upload my source code to a tech giant.

34B should be able to run on 24GiB consumer graphics card, or 32GiB Mac (M1 / M2 chips) with quantization (5~6bit) (and 7B should be able to run on your smart toaster).

Re: Code Llama, a state-of-the-art large language model for coding

#175

Post by dangelov »

Draiken wrote: post #166

As a complete noob at actually running these models, what kind of hardware are we talking here? Couldn't pick that up from the README. I absolutely love the idea of using one of these models without having to upload my source code to a tech giant.

I've used Ollama to run Llama 2 (all variants) on my 2020 Intel MacBook Pro - it's incredibly easy. You just install the app and run a couple of shell commands. I'm guessing soon-ish this model will be available too and then you'd be able to use it with the Continue VS Code extension.

Edited to add: Though somewhat slow, swap seems to have been a good enough replacement for not having the loads of RAM required. Ollama says "32 GB to run the 13B models", but I'm running the llama2:13b model on a 16 GB MBP.

Re: Code Llama, a state-of-the-art large language model for coding

#176

Post by artninja1988 »

smoldesu wrote: post #118
emmender1 wrote: post #49

Earlier quoted context omitted.

the only beneficiary of this are the hardware vendors.. nvidia and amd. and startups which get these foundation models for free. because language models are a complementary product, and the complement must be commoditized as a strategy. I see AMD as a bigger beneficiary, since, very soon, amd will equal nvidia for inference and fine-tuning, but amd has a long way to go to equal in foundation model training.

> and startups which get these foundation models for free. It's licensed non-commercially, so I'm not sure what those startups stand to gain. > since, very soon, amd will equal nvidia for inference and fine-tuning Source? If you're referring to Olive, it is indeed impressive but also has caveats: 1. It is just as proprietary as CUDA or CoreML. 2. You need a copy of Windows and licensed DirectX to use those optimizati…

Llama2 is not licensed non-commercially. There was some weird provisions about not having more than 1 billion users at llama launch though

Re: Code Llama, a state-of-the-art large language model for coding

#177

Post by ilaksh »

jonchurch_ wrote: post #163
nabakin wrote: post #138

Earlier quoted context omitted.

Looks like they aren't releasing a pretty interesting model too. In the paper they mention a "Unnatural Code Llama" which wipes the floor with every other model/finetune on every benchmark except for slightly losing to Code Llama Python on MBPP pass@100 and slightly losing to GPT-4 on HumanEval pass@1 which is insane. Meta says later on that they aren't releasing it and give no explanation. I wonder why given how inc…

The paper states it was instruction fine tuned with synthetic data (LLM generated instructions) ala another paper (“Unnatural Instructions: Tuning Language Models with (Almost) No Human Labor”). The github repo associated with that paper is linked below. It links to the paper on arxiv, but also has some data in the repo. https://github.com/orhonovich/unnatural-instructions

Maybe they used GPT-4 to train it. OpenAI terms of use don't allow that to be released commercially.

Re: Code Llama, a state-of-the-art large language model for coding

#178

Post by bryanlyon »

Llama is a very cool language model, it being used for coding was all but inevitable. I especially love it being released open for everyone.

I do wonder about how much use it'll get, seeing as running a heavy language model on local hardware is kinda unlikely for most developers. Not everyone is runnning a system powerful enough to equip big AIs like this. I also doubt that companies are going to set up large AIs for their devs. It's just a weird positioning.

Re: Code Llama, a state-of-the-art large language model for coding

#179

Post by Der_Einzige »

vessenes wrote: post #34
lolinder wrote: post #7

Does anyone have a good explanation for Meta's strategy with AI? The only thing I've been able to think is they're trying to commoditize this new category before Microsoft and Google can lock it in, but where to from there? Is it just to block the others from a new revenue source, or do they have a longer game they're playing?

They are behind commercially, very behind. They also don't have the same economic setup and DNA as MS/OpenAI. Large corporate customers don't pay for access to the FB cloud, nor are they likely to -- Ellison has spent years building out Oracle Cloud, and he's on the FB board, for example. And I bet you didn't think of using Oracle's Cloud for your last project. So, your company DNA is free-to-all social based on ad m…

You ought to think about using Oracle Cloud for your next LLM/GPU project, because they sell access to A100/H100s for cheap and they actually have them in stock!

Re: Code Llama, a state-of-the-art large language model for coding

#180

Post by EvgeniyZh »

nabakin wrote: post #138
redox99 wrote: post #38

The highlight IMO > The Code Llama models provide stable generations with up to 100,000 tokens of context. All models are trained on sequences of 16,000 tokens and show improvements on inputs with up to 100,000 tokens. Edit: Reading the paper, key retrieval accuracy really deteriorates after 16k tokens, so it remains to be seen how useful the 100k context is.

Looks like they aren't releasing a pretty interesting model too. In the paper they mention a "Unnatural Code Llama" which wipes the floor with every other model/finetune on every benchmark except for slightly losing to Code Llama Python on MBPP pass@100 and slightly losing to GPT-4 on HumanEval pass@1 which is insane. Meta says later on that they aren't releasing it and give no explanation. I wonder why given how inc…

Note that current GPT-4 pass@1 for HumanEval is closer to 90% than to 67% reported in GPT-4 technical report, as reported, e.g., in [1]

[1] https://arxiv.org/abs/2305.01210

Post reply on HN