Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Code Llama, a state-of-the-art large language model for coding

ai.meta.com

Post reply on HN
Print view View original

131–140 of 525 posts

Re: Code Llama, a state-of-the-art large language model for coding

#131

Post by jebarker »

ilaksh wrote: post #61
naillo wrote: post #47

Feels like we're like a year away from local LLMs that can debug code reliably (via being hooked into console error output as well) which will be quite the exciting day.

Have you tried Code Llama? How do you know it can't do it already? In my applications, GPT-4 connected to a VM or SQL engine can and does debug code when given error messages. "Reliably" is very subjective. The main problem I have seen is that it can be stubborn about trying to use outdated APIs and it's not easy to give it a search result with the correct API. But with a good web search and up to date APIs, it can d…

What does "GPT-4 connected to a VM or SQL engine" mean?

Re: Code Llama, a state-of-the-art large language model for coding

#132

Post by mymac »

ttul wrote: post #130
mymac wrote: post #122

Never before in the history of mankind was a group so absolutely besotted with the idea of putting themselves out of a job.

That’s just one perspective… Another perspective is that LLMs enable programmers to skip a lot of the routine and boring aspects of coding - looking up stuff, essentially - so they can focus on the fun parts that engage creativity.

But it won't stop there. Why would it stop at some arbitrarily defined boundary? The savings associated with no longer having to pay programmers the amounts of money that they believe they are worth (high enough to result in collusion between employers) are just too tempting.

Re: Code Llama, a state-of-the-art large language model for coding

#133

Post by artificialLimbs »

lordnacho wrote: post #128

Copilot has been working great for me thus far, but it's limited by its interface. It seems like it only knows how to make predictions for the next bit of text. Is anyone working on a code AI that can suggest refactorings? "You should pull these lines into a function, it's repetitive" "You should change this structure so it is easier to use" Etc

I let mine generate whatever it likes, then add a comment below such as "# Refactor the above to foo.." Works fairly well at times.

Re: Code Llama, a state-of-the-art large language model for coding

#134

Post by make3 »

lordnacho wrote: post #128

Copilot has been working great for me thus far, but it's limited by its interface. It seems like it only knows how to make predictions for the next bit of text. Is anyone working on a code AI that can suggest refactorings? "You should pull these lines into a function, it's repetitive" "You should change this structure so it is easier to use" Etc

There's an instruct model in there, you can definitely use it for this, that's one of the objectives.

An instruct model means that you can ask it to do what you want, including asking it to give you refactoring ideas from the code you will give it.

Re: Code Llama, a state-of-the-art large language model for coding

#135

Post by phillipcarter »

lordnacho wrote: post #128

Copilot has been working great for me thus far, but it's limited by its interface. It seems like it only knows how to make predictions for the next bit of text. Is anyone working on a code AI that can suggest refactorings? "You should pull these lines into a function, it's repetitive" "You should change this structure so it is easier to use" Etc

SourceGraph Cody is going in that direction, as is Copilot Chat. But it's still early days. I don't think there's anything robust here yet.

Re: Code Llama, a state-of-the-art large language model for coding

#136

Post by redox99 »

modeless wrote: post #117
redox99 wrote: post #107

Earlier quoted context omitted.

I can't imagine it being better than Llama1 33B, after all this code finetuning.

But the license for llama 2 is a whole lot better.

Meh.

If you're using it commercially you're probably deploying it on a server where you're not limited by the 24GB and you can just run llama 2 70b.

The majority of people who want to run it locally on 24GB either want roleplay (so non commercial) or code (you have codellama)

Re: Code Llama, a state-of-the-art large language model for coding

#137

Post by fudged71 »

modeless wrote: post #115
scriptsmith wrote: post #72

How are people using these local code models? I would much prefer using these in-context in an editor, but most of them seem to be deployed just in an instruction context. There's a lot of value to not having to context switch, or have a conversation. I see the GitHub copilot extensions gets a new release one every few days, so is it just that the way they're integrated is more complicated so not worth the effort?

http://cursor.sh integrates GPT-4 into vscode in a sensible way. Just swapping this in place of GPT-4 would likely work perfectly. Has anyone cloned the OpenAI HTTP API yet?

I was tasked with a massive project over the last month and I'm not sure I could have done it as fast as I have without Cursor. Also check out the Warp terminal replacement. Together it's a winning combo!

Re: Code Llama, a state-of-the-art large language model for coding

#138

Post by nabakin »

redox99 wrote: post #38

The highlight IMO > The Code Llama models provide stable generations with up to 100,000 tokens of context. All models are trained on sequences of 16,000 tokens and show improvements on inputs with up to 100,000 tokens. Edit: Reading the paper, key retrieval accuracy really deteriorates after 16k tokens, so it remains to be seen how useful the 100k context is.

Looks like they aren't releasing a pretty interesting model too. In the paper they mention a "Unnatural Code Llama" which wipes the floor with every other model/finetune on every benchmark except for slightly losing to Code Llama Python on MBPP pass@100 and slightly losing to GPT-4 on HumanEval pass@1 which is insane.

Meta says later on that they aren't releasing it and give no explanation. I wonder why given how incredible it seems to be.

Re: Code Llama, a state-of-the-art large language model for coding

#139

Post by natch »

Why wouldn’t they provide a hosted version? Seems like a no brainer… they have the money, the hardware, the bandwidth, the people to build support for it, and they could design the experience and gather more learning data about usage in the initial stages, while putting a dent in ChatGPT commercial prospects, and all while still letting others host and use it elsewhere. I don’t get it. Maybe it was just the fastest option?

Re: Code Llama, a state-of-the-art large language model for coding

#140

Post by chaorace »

gw67 wrote: post #2

In your opinion, Why Meta does this?

To a certain extent, I think it's just IBM disease. A company the size of Meta is expected to have an AI research department like Microsoft or Google, even if their core business (social media) derives relatively less benefit from the technology.

Pretend you're an uncreative PM on an AI team; what part of Facebook or VR could you feasibly improve by iterating on LLMs? Perhaps the content moderation system... but that would require wrangling with the company ethics comittee and someone else at the company probably already took ownership that idea. You've gotta do something compelling or else your ML engineers are going to run off somewhere else.

If I were to ask my ML engineers about what they wanted to work on, they're going to avoid areas where their model is outgunned (i.e.: chat) and instead prefer lower hanging fruit which generalizes well on a resume (i.e.: "Pioneered and published key innovations in LLM code-generation").

Of course, the alternative answer is that Meta wants to replace all of their jr. developers with GPUs, but I think their leadership is a little too preoccupied with VR to be actively pushing for such a transformative initiative in anything more than a very uninvested capacity (e.g.: "Sure I'll greenlight this. Even if it doesn't pay off I don't have any better ideas")

Post reply on HN