Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Code Llama, a state-of-the-art large language model for coding

ai.meta.com

Post reply on HN
Print view View original

151–160 of 525 posts

Re: Code Llama, a state-of-the-art large language model for coding

#151

Post by redox99 »

natch wrote: post #139

Why wouldn’t they provide a hosted version? Seems like a no brainer… they have the money, the hardware, the bandwidth, the people to build support for it, and they could design the experience and gather more learning data about usage in the initial stages, while putting a dent in ChatGPT commercial prospects, and all while still letting others host and use it elsewhere. I don’t get it. Maybe it was just the fastest o…

Probably the researchers at meta are only interested in research, and productionizing this would be up to other teams.

Re: Code Llama, a state-of-the-art large language model for coding

#152

Post by adocomplete »

lordnacho wrote: post #128

Copilot has been working great for me thus far, but it's limited by its interface. It seems like it only knows how to make predictions for the next bit of text. Is anyone working on a code AI that can suggest refactorings? "You should pull these lines into a function, it's repetitive" "You should change this structure so it is easier to use" Etc

Give Cody a try! (Cody.dev)

With Cody you can create embeddings for your entire repo, so Cody will have much greater context about your code base and the problems you're trying to solve.

Disclaimer: I just joined Sourcegraph a few weeks ago.

Re: Code Llama, a state-of-the-art large language model for coding

#153

Post by regularfry »

make3 wrote: post #134
lordnacho wrote: post #128

Copilot has been working great for me thus far, but it's limited by its interface. It seems like it only knows how to make predictions for the next bit of text. Is anyone working on a code AI that can suggest refactorings? "You should pull these lines into a function, it's repetitive" "You should change this structure so it is easier to use" Etc

There's an instruct model in there, you can definitely use it for this, that's one of the objectives. An instruct model means that you can ask it to do what you want, including asking it to give you refactoring ideas from the code you will give it.

Sounds like what's needed is a bit of tooling in the background consistently asking the LLM "How would you improve this code?" so you don't need to actually ask it.

Re: Code Llama, a state-of-the-art large language model for coding

#154

Post by Draiken »

dangerwill wrote: post #103

It's really sad how everyone here is fawning over tech that will destroy you own livelihoods. "AI won't take your job, those who use AI will" is purely short term, myopic thinking. These tools are not aimed to help workers, the end goal is to make it so you don't need to be an engineer to build software, just let the project manager or director describe the system they want and boom there it is. You can scream that t…

I agree. This will ultimately become another way to extract more value straight into the pockets of the owners.

Unfortunately, I don't believe there's a way to stop (or even slow down) this train. We can't defeat it, so the only logical answer is to join it.

It's the classical issue with progress removing jobs. In today's world, since mostly everyone (aside from the capitalists themselves) relies on jobs to survive, barring a complete switch from capitalism (which will not happen in our lifetimes), we're fucked.

Next best thing we can do is to try and democratize it enough so that not only the rich have access to it.

Re: Code Llama, a state-of-the-art large language model for coding

#155

Post by armchairhacker »

lordnacho wrote: post #128

Copilot has been working great for me thus far, but it's limited by its interface. It seems like it only knows how to make predictions for the next bit of text. Is anyone working on a code AI that can suggest refactorings? "You should pull these lines into a function, it's repetitive" "You should change this structure so it is easier to use" Etc

Copilot calls these "Code Brushes" https://githubnext.com/projects/code-brushes/

Last I heard they are in beta and don't work very well (even on the examples page: the "add types" brush is too strict, since `a` and `b` are checked for `null`, and the "fix simple bug" is a typo)

Re: Code Llama, a state-of-the-art large language model for coding

#156

Post by hutzlibu »

vessenes wrote: post #34
lolinder wrote: post #7

Does anyone have a good explanation for Meta's strategy with AI? The only thing I've been able to think is they're trying to commoditize this new category before Microsoft and Google can lock it in, but where to from there? Is it just to block the others from a new revenue source, or do they have a longer game they're playing?

They are behind commercially, very behind. They also don't have the same economic setup and DNA as MS/OpenAI. Large corporate customers don't pay for access to the FB cloud, nor are they likely to -- Ellison has spent years building out Oracle Cloud, and he's on the FB board, for example. And I bet you didn't think of using Oracle's Cloud for your last project. So, your company DNA is free-to-all social based on ad m…

"b) this AI thing is huge."

Yeah, there are tons of opportunities for AI to do something with facebooks private user data and sell new services. For users to create engagement - and for ad companies to get very good targeted ads delivered. It is of course a challenge, to update the models on the fly, to include the latest private data, but then you can tailor an ad, that has subtil references to the latest shared wishes of the user. Probably quite effective.

So for now they mainly need top talent, to make some of it work. And open source is the best bet, for creating a ecosystem they can control and get talents who already trained on their tools. And they loose allmost nothing, because yes, they ain't in the cloud buisness.

So I will continue to not use facebook. But the models I will try.

Re: Code Llama, a state-of-the-art large language model for coding

#157

Post by msp26 »

thewataccount wrote: post #124
scriptsmith wrote: post #72

How are people using these local code models? I would much prefer using these in-context in an editor, but most of them seem to be deployed just in an instruction context. There's a lot of value to not having to context switch, or have a conversation. I see the GitHub copilot extensions gets a new release one every few days, so is it just that the way they're integrated is more complicated so not worth the effort?

For in-editor like copilot you can try this locally - https://github.com/smallcloudai/refact This works well for me except the 15B+ don't run fast enough on a 4090 - hopefully exllama supports non-llama models, or maybe it'll support CodeLLaMa already I'm not sure. For general chat testing/usage this works pretty well with lots of options - https://github.com/oobabooga/text-generation-webui/

>This works well for me except the 15B+ don't run fast enough on a 4090

I assume quantized models will run a lot better. TheBloke already seems like he's on it.

https://huggingface.co/TheBloke/CodeLlama-13B-fp16

Re: Code Llama, a state-of-the-art large language model for coding

#158

Post by jmiskovic »

esperent wrote: post #27
reacharavindh wrote: post #4

Code llama Python is very interesting. Specifically tuned for Python. I wonder if we could make such specific LLMs (one that is proficient in all things Rust, another- all things Linux, all things genomics, all things physics modeling etc) and have them talk to each other to collaboratively solve problems. That would be a crazy future thing! Putting machines truly to work..

I think this is called "mixture of experts" and also there's a lot of speculation that it's how GPT-4 works, although probably with just a few large models rather than many small ones.

It's been confirmed by multiple (unofficial) sources that GPT-4 is 8 models, each 220B parameters. Another rumor is GPT-4 being 16x111B models.

There's a quite fresh and active project replicating something similar with herd of llamas: https://github.com/jondurbin/airoboros

Re: Code Llama, a state-of-the-art large language model for coding

#159

Post by criley2 »

rvnx wrote: post #98

Amazing! It's great that Meta is making AI progress. In the meantime, we are still waiting for Google to show what they have (according to their research papers, they are beating others). > User: Write a loop in Python that displays the top 10 prime numbers. > Bard: Sorry I am just an AI, I can't help you with coding. > User: How to ask confirmation before deleting a file ? > Bard: To ask confirmation before deleting…

I don't get comments like this, we can all go and test Bard and see that what you're saying isn't true

https://g.co/bard/share/95761dd6d45e

Re: Code Llama, a state-of-the-art large language model for coding

#160

Post by lsmeducation »

GreedClarifies wrote: post #113
doomlaser wrote: post #100

Earlier quoted context omitted.

I watched a good talk from Yann LeCun who is Chief AI Scientist at Meta, and he explained that the thinking is that open source AI models will be the long-term winner, so it's best for them to work in that arena. https://www.youtube.com/watch?v=vyqXLJsmsrk

That's not a business strategy. Likely this is driven by ego. Yann wants to cement his position as a leader in AI and while he clearly does not appreciate LLMS at all, he realizes that he needs to make waves in this area. Mark needs a generative product and has invested tremendously in the infrastructure for AI in general (for recommendation). He needs researchers to use that infrastructure to create a generative pro…

Linux is wasn’t a business strategy either.
Post reply on HN