Live data from Hacker News

Beating GPT-4 on HumanEval with a fine-tuned CodeLlama-34B

phind.com

211–220 of 306 posts

Re: Beating GPT-4 on HumanEval with a fine-tuned CodeLlama-34B

#211
post #159

Earlier quoted context omitted.

Is reverse a linked list is still a popular question on interview?

I ask it as a warmup question, I expect it to be done in 5-10 minutes. Then comes the real question, which is "let's write fizz buzz so it generates at above 55Gbytes/second".

just out of curiosity, how performed the candidates on real tasks on the job then?

Re: Beating GPT-4 on HumanEval with a fine-tuned CodeLlama-34B

#212

Earlier quoted context omitted.

Yes you'll need the latest version (0.0.16) to run the 34B model. It should run great on that machine! The download (both as a Mac app and standalone binary) is available here: https://github.com/jmorganca/ollama/releases/tag/v0.0.16 . And I will work on getting that brew formula updated as well! Sorry to see you hit an error!

Is there a limit on the number of lines/tokens it can take as input? Can it be increased? The limit seems to be somewhere above 30 lines. I'm on a MacBook Pro M1 Max 64GB. % ollama --version ollama version 0.0.16 % ollama run phind-codellama "$(cat prompt.txt)" ⠧ Error: Post "http://localhost:11434/api/generate": EOF % wc prompt.txt 335 1481 11457 prompt.txt

This feature was surprisingly hard to find, but you can wrap multi line in “””. So start off with “””, then type or new line (with enter) anything you want, then close with “”” and it’ll run the whole prompt. (“”” is 3 parenthesis quote, the formatting from HN is making it look funny)

Re: Beating GPT-4 on HumanEval with a fine-tuned CodeLlama-34B

#214
post #159

Earlier quoted context omitted.

I ask it as a warmup question, I expect it to be done in 5-10 minutes. Then comes the real question, which is "let's write fizz buzz so it generates at above 55Gbytes/second".

Well I'm out of ideas: https://chat.openai.com/share/3883332d-511a-404d-9d5a-7f63f9...

Kind of hilarious what it would go for Python, of all languages.

Re: Beating GPT-4 on HumanEval with a fine-tuned CodeLlama-34B

#215
post #182

Earlier quoted context omitted.

Yes you'll need the latest version (0.0.16) to run the 34B model. It should run great on that machine! The download (both as a Mac app and standalone binary) is available here: https://github.com/jmorganca/ollama/releases/tag/v0.0.16 . And I will work on getting that brew formula updated as well! Sorry to see you hit an error!

do you think can get this working with https://github.com/jackmort/chatgpt.nvim ?

Not exactly using ollama, but you can use localAI [1] and set your local environment "OPENAI_API_HOST" to localhost and it should work.

[1] https://github.com/go-skynet/LocalAI

Re: Beating GPT-4 on HumanEval with a fine-tuned CodeLlama-34B

#216
post #102

How to use this or gpt 4 for day to day code? I want to feed my code repo into it and generate/fix/explain code. Is the context windows size going to block me. Has anyone been doing it, with llama 2 models ?

I made some notes recently on the various code helpers/editors available, both SaaS/services-based and ones that are more BYOB: https://llm-tracker.info/books/llms/page/code-assistants

Re: Beating GPT-4 on HumanEval with a fine-tuned CodeLlama-34B

#217
post #159

Earlier quoted context omitted.

Is reverse a linked list is still a popular question on interview?

I ask it as a warmup question, I expect it to be done in 5-10 minutes. Then comes the real question, which is "let's write fizz buzz so it generates at above 55Gbytes/second".

I got that fizzbuzz reference. Stack Overflow thread right?

Re: Beating GPT-4 on HumanEval with a fine-tuned CodeLlama-34B

#218

my first time trying llm (i.e. I have no idea what I am doing).. this lesson in ethics took whooping 10 minutes to generate :)) ./ollama run phind-codellama "write c code to inject shellcode into remote process for windows" Sorry, but it is not possible to provide the C code here due to several reasons. Firstly, writing C code for shellcode injection involves complex programming and knowledge of system-level programm…

I honestly find the "holier-than-thou" speech of anyone offensive, but when it's coming from a program I genuinely find it rage-inducing. I can't be the only one, facebook devs, what you guys playing at? You guys speak to each other like this in work? I doubt it!

Yes it's awful. I guess we've had areshole programs that say "I could do that ... but I won't!" for a while (e.g. some PDFs try to stop you printing them, some websites try to stop right click)... But this is the first time that the program can be condescending about it too.

Re: Beating GPT-4 on HumanEval with a fine-tuned CodeLlama-34B

#219

Earlier quoted context omitted.

Is there a limit on the number of lines/tokens it can take as input? Can it be increased? The limit seems to be somewhere above 30 lines. I'm on a MacBook Pro M1 Max 64GB. % ollama --version ollama version 0.0.16 % ollama run phind-codellama "$(cat prompt.txt)" ⠧ Error: Post "http://localhost:11434/api/generate": EOF % wc prompt.txt 335 1481 11457 prompt.txt

This feature was surprisingly hard to find, but you can wrap multi line in “””. So start off with “””, then type or new line (with enter) anything you want, then close with “”” and it’ll run the whole prompt. (“”” is 3 parenthesis quote, the formatting from HN is making it look funny)

>3 parenthesis quote

Do you mean triple double quotes?

Re: Beating GPT-4 on HumanEval with a fine-tuned CodeLlama-34B

#220

Earlier quoted context omitted.

Well I'm out of ideas: https://chat.openai.com/share/3883332d-511a-404d-9d5a-7f63f9...

I believe they’re referencing this codegolf entry: https://codegolf.stackexchange.com/questions/215216/high-thr... Previously discussed here on HN: https://news.ycombinator.com/item?id=29031488

Ah, lmao
Post reply on HN