Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Code Llama, a state-of-the-art large language model for coding

ai.meta.com

Post reply on HN
Print view View original

121–130 of 525 posts

Re: Code Llama, a state-of-the-art large language model for coding

#121

Post by ohyes »

ramesh31 wrote: post #24
up6w6 wrote: post #11

Even the 7B model of code llama seems to be competitive with Codex, the model behind copilot https://ai.meta.com/blog/code-llama-large-language-model-cod...

>Even the 7B model of code llama seems to be competitive with Codex, the model behind copilot It's extremely good. I keep a terminal tab open with 7b running for all of my "how do I do this random thing" questions while coding. It's pretty much replaced Google/SO for me.

What hardware do you have that lets you run 7b and do other stuff at the same time?

Re: Code Llama, a state-of-the-art large language model for coding

#123

Post by ttul »

malwrar wrote: post #119
snippyhollow wrote: post #96

Earlier quoted context omitted.

We changed RoPE's theta from 10k to 1m and fine-tuned with 16k tokens long sequences.

Curious, what led you to adjusting the parameters this way? Also, have you guys experimented with ALiBi[1] which claims better extrapolative results than rotary positional encoding? [1]: https://arxiv.org/abs/2108.12409 (charts on page two if you’re skimming)

Undoubtedly, they have tried ALiBi…

Re: Code Llama, a state-of-the-art large language model for coding

#124

Post by thewataccount »

scriptsmith wrote: post #72

How are people using these local code models? I would much prefer using these in-context in an editor, but most of them seem to be deployed just in an instruction context. There's a lot of value to not having to context switch, or have a conversation. I see the GitHub copilot extensions gets a new release one every few days, so is it just that the way they're integrated is more complicated so not worth the effort?

For in-editor like copilot you can try this locally - https://github.com/smallcloudai/refact

This works well for me except the 15B+ don't run fast enough on a 4090 - hopefully exllama supports non-llama models, or maybe it'll support CodeLLaMa already I'm not sure.

For general chat testing/usage this works pretty well with lots of options - https://github.com/oobabooga/text-generation-webui/

Re: Code Llama, a state-of-the-art large language model for coding

#125

Post by bbor »

dangerwill wrote: post #103

It's really sad how everyone here is fawning over tech that will destroy you own livelihoods. "AI won't take your job, those who use AI will" is purely short term, myopic thinking. These tools are not aimed to help workers, the end goal is to make it so you don't need to be an engineer to build software, just let the project manager or director describe the system they want and boom there it is. You can scream that t…

We have three options, IMO:

1. As a species decide to never build another LLM, ever.

2. Change the path of society from the unequal, capitalist one it’s taken the last 2-300 years.

3. Give up

I know which I believe in :). Do you disagree?

Re: Code Llama, a state-of-the-art large language model for coding

#126

Post by hmottestad »

ohyes wrote: post #121
ramesh31 wrote: post #24

Earlier quoted context omitted.

>Even the 7B model of code llama seems to be competitive with Codex, the model behind copilot It's extremely good. I keep a terminal tab open with 7b running for all of my "how do I do this random thing" questions while coding. It's pretty much replaced Google/SO for me.

What hardware do you have that lets you run 7b and do other stuff at the same time?

Maybe a MacBook Pro. The Apple silicon chops can offload a special AI inference engine, and all ram is accessible by all parts of the chip.

Re: Code Llama, a state-of-the-art large language model for coding

#127

Post by andrewjl »

What I found interesting in Meta's paper is the mention of HumanEval[1] and MBPP[2] as benchmarks for code quality. (Admittedly maybe they're well-known to those working in the field.)

I haven't yet read the whole paper (nor have I looked at the benchmark docs which might very well cover this) but curious how these are designed to avoid issues with overfitting. My thinking here is that canned algorithm type problems common in software engineering interviews are probably over represented in the training data used for these models. Which might point to artificially better performance by LLMs versus their performance on more domain-specific type tasks they might be used for in day-to-day work.

[1] https://github.com/openai/human-eval

[2] https://github.com/google-research/google-research/tree/mast...

Re: Code Llama, a state-of-the-art large language model for coding

#128

Post by lordnacho »

Copilot has been working great for me thus far, but it's limited by its interface. It seems like it only knows how to make predictions for the next bit of text.

Is anyone working on a code AI that can suggest refactorings?

"You should pull these lines into a function, it's repetitive"

"You should change this structure so it is easier to use"

Etc

Re: Code Llama, a state-of-the-art large language model for coding

#129

Post by nabakin »

modeless wrote: post #8

Interesting that there's a 34B model. That was missing from the original Llama 2 release. I wonder if it's still usable for general non-code chat tasks or if the code fine tuning destroyed that. It should be the best model that would still fit on 24GB gaming GPUs with quantization, because 70B doesn't fit.

Looks like they left out another model though. In the paper they mention a "Unnatural Code Llama" which wipes the floor with every other model/finetune on every benchmark except for slightly losing to Code Llama Python on MBPP pass@100 and slightly losing to GPT-4 on HumanEval pass@1 which is insane.

Meta says later on that they aren't releasing it and give no explanation. I wonder why given how incredible it seems to be.

Re: Code Llama, a state-of-the-art large language model for coding

#130

Post by ttul »

mymac wrote: post #122

Never before in the history of mankind was a group so absolutely besotted with the idea of putting themselves out of a job.

That’s just one perspective… Another perspective is that LLMs enable programmers to skip a lot of the routine and boring aspects of coding - looking up stuff, essentially - so they can focus on the fun parts that engage creativity.
Post reply on HN