I can't seem to find a demo, if anyone has a chance to test it, how does it compare to replit and wizardcoder?
StableCode
11–20 of 109 posts
Re: StableCode
#12I can't seem to find a demo, if anyone has a chance to test it, how does it compare to replit and wizardcoder?
Looks like there is one on the Hugging Face page: https://huggingface.co/stabilityai/stablecode-instruct-alpha...
Not very promising based on this lame test
Re: StableCode
#13Hard to believe it can work that well when it only has 3B parameters, but I'd love to be proven wrong.
I was impressed enough by replit's 2.7B model that I'm convinced it's doable. I have a 4090 and consider that the "max expected card for a consumer to own". Also exllama doesn't support non-llama models and the creator doesn't seem interested in adding support for wizardcoder/etc. Because of this, using the alternatives are prohibitively slow to use a quantized 16B model on a 4090 (if the exllama author reads this _p…
I'm fairly confident a coding specific model should be a lot smaller - 3b should be plenty if not 1b or less. As it stands, there are quite a few 7-13b model sizes that can predict natural language quite well. Code seems at its surface a much simpler language, strict grammars, etc so I wouldn't think it needs to be anywhere near as large as the nlp models. Right now people are retraining nlp models to work with code, but I think the best code helper models in the future will be trained primarily on code and maybe fine tuned on some language. I'm thinking less of a chat bot api and more of a giant leap in "intellisense" services.
Re: StableCode
#14Is this a "product" that one could install and use or a model that one should expect an OEM to integrate into a product before programmers can use it? I'm asking because I don't see any links that would help me figure out how to try it out.
Yes, the model is available. However, it just released so no one has wrapped it in a plugin yet. I would expect that within the month there will be a nicely runnable local version, similar to llama2's wrappers.
Re: StableCode
#15Hard to believe it can work that well when it only has 3B parameters, but I'd love to be proven wrong.
phi-1[0] is only 1.3 billion parameters and performs very well in coding tasks - small models have a massive amount of potential
Re: StableCode
#16Is this a "product" that one could install and use or a model that one should expect an OEM to integrate into a product before programmers can use it? I'm asking because I don't see any links that would help me figure out how to try it out.
Ctrl-F for “Code for using StableCode Instruct to generate a response to a given instruction.” and you’ll see a super straightforward piece of code to copy to test it out for generating code
Re: StableCode
#17Hard to believe it can work that well when it only has 3B parameters, but I'd love to be proven wrong.
Reminder that GPT-2 was considered “too dangerous” to be released at just 1.5B weights
Re: StableCode
#18I love stability AI
Re: StableCode
#19Earlier quoted context omitted.
I was impressed enough by replit's 2.7B model that I'm convinced it's doable. I have a 4090 and consider that the "max expected card for a consumer to own". Also exllama doesn't support non-llama models and the creator doesn't seem interested in adding support for wizardcoder/etc. Because of this, using the alternatives are prohibitively slow to use a quantized 16B model on a 4090 (if the exllama author reads this _p…
I'm fairly confident a coding specific model should be a lot smaller - 3b should be plenty if not 1b or less. As it stands, there are quite a few 7-13b model sizes that can predict natural language quite well. Code seems at its surface a much simpler language, strict grammars, etc so I wouldn't think it needs to be anywhere near as large as the nlp models. Right now people are retraining nlp models to work with code,…
I'd really like to see smaller models trained on only one specific language, with it's own language specific tokenizer. I imagine the reduction in vocab size would translate to handling more context easier?
Re: StableCode
#20Either way, the race to zero has been further accelerated. Stability AI, Apple, Meta, etc are clearly at the finish line putting pressure on cloud only AI models and cannot raise prices or compete with free.
> Stability AI, Apple, Meta, etc are clearly at the finish line I'm very optimistic and expect them to catch up. I've used the open models a lot, to be clear they are starting to compare to GPT3.5Turbo right now, they can't compete with GPT4 at all. GPT4 is almost a year old from when it finished training I think? I expect open source models to stay ~1.5 years behind. That said they will eventually be "good enough".…
> Keep in mind too though that using and scaling GPUs is not free. You have to run the models somewhere.
Long or medium term these will probably be dirt cheap to just run in the background though. It might be within 3-5 years since parallel compute is still growing and isn’t as bounded by moores law stagnation