Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Code Llama, a state-of-the-art large language model for coding

ai.meta.com

Post reply on HN
Print view View original

321–330 of 525 posts

Re: Code Llama, a state-of-the-art large language model for coding

#321

Post by throwuxiytayq »

blibble wrote: post #275
daemonologist wrote: post #236

Works nearly out of the box with llama.cpp, which makes it easy to try locally: https://github.com/ggerganov/llama.cpp/issues/2766 Here's some output from q4_0 quantization of CodeLlama-7b-Python (first four lines are the prompt): # prints the first ten prime numbers def print_primes(): i = 2 num_printed = 0 # end of prompt while num_printed It will be interesting to see how the larger models perform, especially afte…

I'd fail an interview candidate that suggested adding 1 each time for subsequent prime testing

i’d walk out from an interview that asked me to write a prime number generator

Re: Code Llama, a state-of-the-art large language model for coding

#322

Post by BoorishBears »

julianeon wrote: post #318
Someone1234 wrote: post #315

Business opportunity: I'd pay money for NICE desktop software that can run all these different models (non-subscription, "2-year updates included, then discount pricing" modal perhaps). My wishlist: - Easy plug & play model installation, and trivial to change which model once installed. - Runs a local web server, so I can interact with it via any browser - Ability to feed a model a document or multiple documents and…

What I want is even simpler: just an API that you make requests to and receive answers back. Surprisingly hard to find, outside OpenAI that is.

I'm not into open source LLMs in the slightest, and yet even I've trivially found tools to do what both you and the poster above you wanted

lmstudio actually does what both of you want: provides an easy GUI and serves up your model over a local endpoint that mirrors the OpenAI API.

There's just too much noise in terms of the tooling for LLMs, the solution is fewer higher quality solutions, not more solutions

Re: Code Llama, a state-of-the-art large language model for coding

#323

Post by ingridpan »

julianeon wrote: post #318
Someone1234 wrote: post #315

Business opportunity: I'd pay money for NICE desktop software that can run all these different models (non-subscription, "2-year updates included, then discount pricing" modal perhaps). My wishlist: - Easy plug & play model installation, and trivial to change which model once installed. - Runs a local web server, so I can interact with it via any browser - Ability to feed a model a document or multiple documents and…

What I want is even simpler: just an API that you make requests to and receive answers back. Surprisingly hard to find, outside OpenAI that is.

https://gradient.ai/ is doing that with llama2

Re: Code Llama, a state-of-the-art large language model for coding

#324

Post by gzer0 »

ohyes wrote: post #121
ramesh31 wrote: post #24

Earlier quoted context omitted.

>Even the 7B model of code llama seems to be competitive with Codex, the model behind copilot It's extremely good. I keep a terminal tab open with 7b running for all of my "how do I do this random thing" questions while coding. It's pretty much replaced Google/SO for me.

What hardware do you have that lets you run 7b and do other stuff at the same time?

An M1 Max with 64GB of RAM allows me to run multiple models simultaneously, on top of stable diffusion generating images non-stop + normal chrome, vscode, etc. Definitely feeling the heat, but it's working. Well worth the investment.

Re: Code Llama, a state-of-the-art large language model for coding

#325

Post by jpeterson »

blibble wrote: post #284
droopyEyelids wrote: post #282

Earlier quoted context omitted.

The simple-to-understand, greedy algorithm is always the correct first choice till you have to deal with a constraint.

it's not that though, there's several other typical optimisations in there just not the super obvious one that demonstrates extremely basic understanding of what a prime number is

Having "extremely basic understanding" of prime numbers immediately at one's command is important for approximately 0% of software engineering jobs. If you instant-fail a candidate for this, it says a lot more about you and your organization than the candidate.

Re: Code Llama, a state-of-the-art large language model for coding

#326

Post by csmpltn »

blibble wrote: post #275
daemonologist wrote: post #236

Works nearly out of the box with llama.cpp, which makes it easy to try locally: https://github.com/ggerganov/llama.cpp/issues/2766 Here's some output from q4_0 quantization of CodeLlama-7b-Python (first four lines are the prompt): # prints the first ten prime numbers def print_primes(): i = 2 num_printed = 0 # end of prompt while num_printed It will be interesting to see how the larger models perform, especially afte…

I'd fail an interview candidate that suggested adding 1 each time for subsequent prime testing

> "I'd fail an interview candidate that suggested adding 1 each time for subsequent prime testing"

Congratulations! You must be that arrogant guy everybody hates interviewing with, the one with the superiority complex.

How about instead of just failing people over literally nothing (wasting everybody's time and money) - just ask the candidate whether they could somehow reduce the search space by utilizing the properties of a prime number?

Re: Code Llama, a state-of-the-art large language model for coding

#327

Post by SubiculumCode »

jmorgan wrote: post #316
Someone1234 wrote: post #315

Business opportunity: I'd pay money for NICE desktop software that can run all these different models (non-subscription, "2-year updates included, then discount pricing" modal perhaps). My wishlist: - Easy plug & play model installation, and trivial to change which model once installed. - Runs a local web server, so I can interact with it via any browser - Ability to feed a model a document or multiple documents and…

A few folks and I have been working on an open-source tool that does some of this (and hopefully more soon!) https://github.com/jmorganca/ollama There's a "PrivateGPT" example in there that is similar to your third point above: https://github.com/jmorganca/ollama/tree/main/examples/priva... Would love to know your thoughts

[flagged]

Re: Code Llama, a state-of-the-art large language model for coding

#328

Post by ingridpan »

e12e wrote: post #208

Curious if there are projects to enable working with these things self-hosted, tuned to a git repo as context on the cli, like a Unix filter - or with editors like vim? (I'd love to use this with Helix) I see both vscode and netbeans have a concept of "inference URL" - are there any efforts like language server (lsp) - but for inference?

not quite self-hosted but gradient.ai gives you access to llama2 via CLI

Re: Code Llama, a state-of-the-art large language model for coding

#329

Post by BoorishBears »

j45 wrote: post #212
dangelov wrote: post #175

Earlier quoted context omitted.

I've used Ollama to run Llama 2 (all variants) on my 2020 Intel MacBook Pro - it's incredibly easy. You just install the app and run a couple of shell commands. I'm guessing soon-ish this model will be available too and then you'd be able to use it with the Continue VS Code extension. Edited to add: Though somewhat slow, swap seems to have been a good enough replacement for not having the loads of RAM required. Ollam…

Apple Silicon, especially an M1 Max Studio seems to be an interesting machine to hang on to as the models become more and more efficient with using less and less. If there's nay other opinions or thoughts on this, I'd be very happy to learn as well. I have considered the eGPU route connected to a 1L PC such as a thinkcentre m80/90.

I have a 64 GB M1 Max MBP, and I'd say unless you really have some academic interest towards messing with open models, for now accessing SOTA models via a REST API has better latency for a given quality.

Claude 1.2 instant is as fast as 3.5, follows instructions at a quality closer to 4, and has a 100k context window. Hard to compete with that with an open source model right now.

Re: Code Llama, a state-of-the-art large language model for coding

#330

Post by noduerme »

blibble wrote: post #284
droopyEyelids wrote: post #282

Earlier quoted context omitted.

The simple-to-understand, greedy algorithm is always the correct first choice till you have to deal with a constraint.

it's not that though, there's several other typical optimisations in there just not the super obvious one that demonstrates extremely basic understanding of what a prime number is

I'm mad at myself now that it has eaten 15 minutes of my time trying to come up with the right optimization. What's the trick? 2, +1, and then +2 from there on seems obvious but once you get to 9 is it worth building a list of nonprimes to skip?
Post reply on HN