Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Code Llama, a state-of-the-art large language model for coding

ai.meta.com

Post reply on HN
Print view View original

361–370 of 525 posts

Re: Code Llama, a state-of-the-art large language model for coding

#361

Post by belenos46 »

throwuxiytayq wrote: post #321
blibble wrote: post #275

Earlier quoted context omitted.

I'd fail an interview candidate that suggested adding 1 each time for subsequent prime testing

i’d walk out from an interview that asked me to write a prime number generator

I've done that (maybe it was fizzbuzz, now that I'm thinking about it) and boy howdy does that get the people you're interviewing with agitated. Saying "I'm interviewing for a architect level container orchestration position. If I'm reinventing the wheel writing algorithms, something is terribly wrong" shuts them up, but doesn't make them any happier.

Re: Code Llama, a state-of-the-art large language model for coding

#362

Post by ungruntled »

blibble wrote: post #342
jpeterson wrote: post #325

Earlier quoted context omitted.

Having "extremely basic understanding" of prime numbers immediately at one's command is important for approximately 0% of software engineering jobs. If you instant-fail a candidate for this, it says a lot more about you and your organization than the candidate.

> If you instant-fail a candidate for this, it says a lot more about you and your organization than the candidate. yes, we expect professional software developers to have basic maths skills "what is a prime number" is taught to 7 year olds, it's not vector calculus what else would you consider to be an unreasonable thing for an employer to require? reading and writing skills of a typical 7 year old?

I think the key problem here is that is is a bad programming question. If you know anything about prime numbers then coming up with an answer is trivial. If you expect a more optimized solution, then you are really only gauging the interviewee’s understanding of prime numbers. So effectively the interview is more about mathematics than it is about programming or problem solving.

Re: Code Llama, a state-of-the-art large language model for coding

#363

Post by [deleted] »

blibble wrote: post #275
daemonologist wrote: post #236

Works nearly out of the box with llama.cpp, which makes it easy to try locally: https://github.com/ggerganov/llama.cpp/issues/2766 Here's some output from q4_0 quantization of CodeLlama-7b-Python (first four lines are the prompt): # prints the first ten prime numbers def print_primes(): i = 2 num_printed = 0 # end of prompt while num_printed It will be interesting to see how the larger models perform, especially afte…

I'd fail an interview candidate that suggested adding 1 each time for subsequent prime testing

[deleted]

Re: Code Llama, a state-of-the-art large language model for coding

#364

Post by dontupvoteme »

blibble wrote: post #275
daemonologist wrote: post #236

Works nearly out of the box with llama.cpp, which makes it easy to try locally: https://github.com/ggerganov/llama.cpp/issues/2766 Here's some output from q4_0 quantization of CodeLlama-7b-Python (first four lines are the prompt): # prints the first ten prime numbers def print_primes(): i = 2 num_printed = 0 # end of prompt while num_printed It will be interesting to see how the larger models perform, especially afte…

I'd fail an interview candidate that suggested adding 1 each time for subsequent prime testing

Simply prompting the output with "Optimize " prepended adds your suggestion, and some others.

Re: Code Llama, a state-of-the-art large language model for coding

#365

Post by noduerme »

Our_Benefactors wrote: post #343
noduerme wrote: post #330

Earlier quoted context omitted.

I'm mad at myself now that it has eaten 15 minutes of my time trying to come up with the right optimization. What's the trick? 2, +1, and then +2 from there on seems obvious but once you get to 9 is it worth building a list of nonprimes to skip?

https://stackoverflow.com/a/54544012/1336678 Common approach is to use square roots, this reduces the runtime. Recommend checking out project euler if you like solving hard math-code-o(n)-puzzles.

I didn't want to cheat by looking on S.O. but thanks ;)

Yes it makes sense (in the GPT code) that you'd only go up to i * i ... although looking at pythonic while: statements is just gross to me in this context, it would feel a lot more readable to say, e.g. in PHP:

for ($i=2;$i<sqrt($n);) { $i+=($i==2 ? 1 : 2); //although the first one should just be outside the loop }

Re: Code Llama, a state-of-the-art large language model for coding

#366

Post by [deleted] »

ungruntled wrote: post #362
blibble wrote: post #342

Earlier quoted context omitted.

> If you instant-fail a candidate for this, it says a lot more about you and your organization than the candidate. yes, we expect professional software developers to have basic maths skills "what is a prime number" is taught to 7 year olds, it's not vector calculus what else would you consider to be an unreasonable thing for an employer to require? reading and writing skills of a typical 7 year old?

I think the key problem here is that is is a bad programming question. If you know anything about prime numbers then coming up with an answer is trivial. If you expect a more optimized solution, then you are really only gauging the interviewee’s understanding of prime numbers. So effectively the interview is more about mathematics than it is about programming or problem solving.

[deleted]

Re: Code Llama, a state-of-the-art large language model for coding

#368

Post by noduerme »

simonw wrote: post #360
noduerme wrote: post #320

Earlier quoted context omitted.

Agreed. After several rounds of setting up various python environments and tinkering with directory structures and debugging glitches and quantizing models just to end up playing around for a few minutes and getting bored, it would be nice to have the experience just be seamless. I wouldn't try to set up a workflow around seriously using what's out there to run on localhost now. That said, non-subscription is essenti…

I've been trying to push things in that direction with my LLM tool - the idea is to have Python plugins which you can install that do all of the irritating details to get a model setup. I've not yet been able to solve the challenge of needing CUDA etc for some models though! Plugins so far: https://llm.datasette.io/en/stable/plugins/directory.html

Cool! I've followed your instructions and your blog quite a bit as I've experimented with running local LLMs as well as stable diffusion. It's been especially helpful, as python is not my language or usual environment. Your patience at hacking your way through each new iteration and presenting what's important about them is astonishing; I personally think I'd have gone mad, but you've done great work in charting the territory.

Re: Code Llama, a state-of-the-art large language model for coding

#369

Post by someplaceguy »

d0mine wrote: post #354
daemonologist wrote: post #236

Works nearly out of the box with llama.cpp, which makes it easy to try locally: https://github.com/ggerganov/llama.cpp/issues/2766 Here's some output from q4_0 quantization of CodeLlama-7b-Python (first four lines are the prompt): # prints the first ten prime numbers def print_primes(): i = 2 num_printed = 0 # end of prompt while num_printed It will be interesting to see how the larger models perform, especially afte…

Simple, concise, more efficient: def primes_upto(limit: int): """Generate prime numbers 2 3 5 7 11 13 17 19 23 29

Yeah, but yours was generated by the "post unoptimized code to HN and wait for someone to optimize it" model, which, although free and doesn't require a GPU, is a much slower model.

Re: Code Llama, a state-of-the-art large language model for coding

#370

Post by max51 »

MaKey wrote: post #334
Someone1234 wrote: post #315

Business opportunity: I'd pay money for NICE desktop software that can run all these different models (non-subscription, "2-year updates included, then discount pricing" modal perhaps). My wishlist: - Easy plug & play model installation, and trivial to change which model once installed. - Runs a local web server, so I can interact with it via any browser - Ability to feed a model a document or multiple documents and…

Did you try Oobabooga ( https://github.com/oobabooga/text-generation-webui ) yet?

Oobabooga is a great tool but it still has a long way to go in term of user-friendliness. It's absolutely not plug and play the way that chatgpt is; It requires research, trial and error, and knowledge of the tech to make the model work to its full potential. It's great once you finish setting it up, but it does not compare to what you would expect from a commercial product aimed at normal end-users.

Things like bad default values, no tooltips, an no curated model list to one-click download is what separates a tool like Oobabooga from a paid commercial product. These things require time/money and it would be very unlikely that an open source tool could find resources for all the testing and R&D.

I think there is a big market for products where you pay and can just start chatting with the model without having to ever go to the settings tab or google anything unless you need to do something out of the ordinary.

Post reply on HN