Refact Code LLM: 1.6B LLM for code that reaches 32% HumanEval
1–10 of 111 posts
Re: Refact Code LLM: 1.6B LLM for code that reaches 32% HumanEval
#2We’ve finished training a new code model Refact LLM which took us about a month. The main use-case is for blazing-fast code completion with fill-in-the-middle, additionally, the model could reply to chat prompts.
It has much better performance than all of the code models of similar size, and almost reaches the same HumanEval as Starcoder being 10x smaller in size.
With the small size, it can work with most modern GPUs requiring just 3GB Ram.
You can try self-hosting it in Refact https://github.com/smallcloudai/refact/ and get a local fast copilot alternative with decent suggestions.
Weights and model card https://huggingface.co/smallcloudai/Refact-1_6B-fim.
We would love to hear your feedback!
Re: Refact Code LLM: 1.6B LLM for code that reaches 32% HumanEval
#3That’s an impressive result
The open rail license seems to reference some sort of limitations on safety and unethical use but I can’t see where in the repo that’s spelled out precisely what the authors have in mind?
Re: Refact Code LLM: 1.6B LLM for code that reaches 32% HumanEval
#4We’ve finished training a new code model Refact LLM which took us about a month. The main use-case is for blazing-fast code completion with fill-in-the-middle, additionally, the model could reply to chat prompts. It has much better performance than all of the code models of similar size, and almost reaches the same HumanEval as Starcoder being 10x smaller in size. With the small size, it can work with most modern GPU…
Is it possible to run it as an LSP so that it can be used in editors other than VSCode and JetBrains? (sorry if this question is completely mad, my understanding of how these things work is extremely limited)
Re: Refact Code LLM: 1.6B LLM for code that reaches 32% HumanEval
#5LICENSE
bigscience-openrail-m
https://huggingface.co/smallcloudai/Refact-1_6B-fim/blob/mai...
Re: Refact Code LLM: 1.6B LLM for code that reaches 32% HumanEval
#6Model Stats
- Architecture: LLAMA-like model with multi-query attention
- Objectives Fill-in-the-Middle, Chat
- Tokens context: 4096
- Pretraining tokens: 1.2T
- Finetuning tokens: 40B
- Precision: bfloat16
- GPUs 64 NVidia A5000
- Training time 28 days
Re: Refact Code LLM: 1.6B LLM for code that reaches 32% HumanEval
#7Model Stats - Architecture: LLAMA-like model with multi-query attention - Objectives Fill-in-the-Middle, Chat - Tokens context: 4096 - Pretraining tokens: 1.2T - Finetuning tokens: 40B - Precision: bfloat16 - GPUs 64 NVidia A5000 - Training time 28 days
[deleted]
Re: Refact Code LLM: 1.6B LLM for code that reaches 32% HumanEval
#8LICENSE bigscience-openrail-m https://huggingface.co/smallcloudai/Refact-1_6B-fim/blob/mai...
[deleted]
Re: Refact Code LLM: 1.6B LLM for code that reaches 32% HumanEval
#9For the sake of not giving Microsoft and a few other tech giants immense power over the world, I really do hope the cost and efficiency of LLMs improve dramatically, until we can get GPT-4-equivalent models trained on a few graphics cards and running offline on an iPhone. Really rooting for these kinds of projects until someone makes the breakthrough.
Re: Refact Code LLM: 1.6B LLM for code that reaches 32% HumanEval
#10That’s an impressive result The open rail license seems to reference some sort of limitations on safety and unethical use but I can’t see where in the repo that’s spelled out precisely what the authors have in mind?
[deleted]