Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

CodeAlpaca – Instruction following code generation model

github.com

Post reply on HN
Print view View original

21–30 of 50 posts

Re: CodeAlpaca – Instruction following code generation model

#21

Post by MacsHeadroom »

menzoic wrote: post #12
gigel82 wrote: post #2

The results are pretty good; I wish they'd just publish the models so we can run the inference locally (not too many people have access to 8xA100 to train themselves, though I appreciate including the training data and instructions too).

Anyone with a few hundred bucks to spare can do it by renting GPUs from a cloud provider. It only cost Stanford $600 to create Alpaca from LLAMA. $100 to generate instructions with GPT-3 and $500 to rent cloud GPUs. The license restriction is due to the use of GPT-3 output to train a model.

More like $50 or even $5 or less for the cloud GPUs. Alpaca-7B's compute costs were close to $50 and that was before the 100x cost savings of using LoRA.

A 4bit LoRA fine tune of this project would cost less than $5 to train even up to 30B/33B.

Re: CodeAlpaca – Instruction following code generation model

#22

Post by nickthegreek »

menzoic wrote: post #12
gigel82 wrote: post #2

The results are pretty good; I wish they'd just publish the models so we can run the inference locally (not too many people have access to 8xA100 to train themselves, though I appreciate including the training data and instructions too).

Anyone with a few hundred bucks to spare can do it by renting GPUs from a cloud provider. It only cost Stanford $600 to create Alpaca from LLAMA. $100 to generate instructions with GPT-3 and $500 to rent cloud GPUs. The license restriction is due to the use of GPT-3 output to train a model.

I’d love to see a crowdsourcing platform to donate to specific fine-tuning projects. I would gladly throw some money at someone to do the labor and release the models to the public.

Re: CodeAlpaca – Instruction following code generation model

#23

Post by tarruda »

Didn't explore much, but it seems alpaca-lora has better results for coding tasks. One example I've used was: "Implement quicksort in python.". This is the result with Code alpaca:

    def quicksort(arr):
        if len(arr)  pivot:
                    right_index -= 1
                if left_index
Alpaca lora result:

    def quicksort(arr):
        if len(arr)  pivot]
        return quicksort(left) + [pivot] + quicksort(right)
Shorter and much cleaner, not to mention it works (code alpaca version is broken). Also it matches what ChatGPT generates for me.

Re: CodeAlpaca – Instruction following code generation model

#24

Post by saurik »

dchest wrote: post #18
detrites wrote: post #14

Earlier quoted context omitted.

To avoid the embarrassment of having to serve themselves with a DMCA?

The PR is not in their repository, it's in the fork. Pull Request refers to "pull this from my repository, here's a link", GitHub just presents it in a convenient interface. (But you're right, those who send DMCAs are likely to just a send a link to the original repository :)

One can assuredly delete the pull request from your own list of open pull requests, right? It isn't that the information exists in theory on GitHub: it is that it is still listed right there when you go to check the health of the project's open issues/PRs.

Re: CodeAlpaca – Instruction following code generation model

#25

Post by elcomet »

dchest wrote: post #18
detrites wrote: post #14

Earlier quoted context omitted.

To avoid the embarrassment of having to serve themselves with a DMCA?

The PR is not in their repository, it's in the fork. Pull Request refers to "pull this from my repository, here's a link", GitHub just presents it in a convenient interface. (But you're right, those who send DMCAs are likely to just a send a link to the original repository :)

They could close the PR in their own repo...

Re: CodeAlpaca – Instruction following code generation model

#27

Post by whazor »

politician wrote: post #6
underlines wrote: post #4

"Model weights aren't part of the release for now, to respect OpenAI TOS and LLaMA license." I feel like the whole Open Source ML scene is slowed down by a strong chilling effect. Everyone seems to be afraid to release models. Meanwhile, other models are freely available up to alpaca 30b: https://github.com/underlines/awesome-marketing-datascience/...

I don’t understand why Facebook hasn’t taken down the original magnet links in its own repository.

The people who are maintaining the repository likely have to organise meetings with legal and business teams. When no agreement can be reached, doing nothing is the easiest way forward. They have to do this next to their normal work.

Sending out DCMAs is a different process done, likely done by the legal team.

Re: CodeAlpaca – Instruction following code generation model

#28

Post by [deleted] »

underlines wrote: post #4

"Model weights aren't part of the release for now, to respect OpenAI TOS and LLaMA license." I feel like the whole Open Source ML scene is slowed down by a strong chilling effect. Everyone seems to be afraid to release models. Meanwhile, other models are freely available up to alpaca 30b: https://github.com/underlines/awesome-marketing-datascience/...

[deleted]

Re: CodeAlpaca – Instruction following code generation model

#29

Post by sahil_chaudhary »

tarruda wrote: post #23

Didn't explore much, but it seems alpaca-lora has better results for coding tasks. One example I've used was: "Implement quicksort in python.". This is the result with Code alpaca: def quicksort(arr): if len(arr) pivot: right_index -= 1 if left_index Alpaca lora result: def quicksort(arr): if len(arr) pivot] return quicksort(left) + [pivot] + quicksort(right) Shorter and much cleaner, not to mention it works (code al…

Interesting, in my experiments alpaca-lora performed worse but definitely possible that isn't the case generally.

Re: CodeAlpaca – Instruction following code generation model

#30

Post by sahil_chaudhary »

nico wrote: post #3

> The code runs on a 8xA100 80GB, but can also run on 8xA10040GB or 4xA100 with lower batch size and gradient accumulation steps. To get the GPUs, I suggest using Lambda Labs, best pricing for the best hardware. I wonder how much it was total in $ for the fine-tuning. Also, does anyone have some sort of table/formula that relates MB/GB of training data to $ for fine-tuning?

All included it costs under 70$ for the 13B model. Training 65B now so will report what that will cost.
Post reply on HN