Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

StableCode

stability.ai

Post reply on HN
Print view View original

31–40 of 109 posts

Re: StableCode

#31

Post by miohtama »

3rd3 wrote: post #26

How does it compare to GitHub Copilot?

The model, source, etc. are available under permissive terms

https://huggingface.co/stabilityai/stablecode-instruct-alpha...

You can “run it locally”. Very handy if you do not trust automatically sending all your code to someone in the United States.

Re: StableCode

#32

Post by thorum »

thewataccount wrote: post #8
sebzim4500 wrote: post #2

Hard to believe it can work that well when it only has 3B parameters, but I'd love to be proven wrong.

I was impressed enough by replit's 2.7B model that I'm convinced it's doable. I have a 4090 and consider that the "max expected card for a consumer to own". Also exllama doesn't support non-llama models and the creator doesn't seem interested in adding support for wizardcoder/etc. Because of this, using the alternatives are prohibitively slow to use a quantized 16B model on a 4090 (if the exllama author reads this _p…

replit’s model is surprisingly good at generating code, even at following complex instructions that I was sure would confuse it. I have found it’s a bit weak on code analysis, for open-ended questions like ‘is there a bug anywhere in this code?’ that GPT-4 can answer.

Re: StableCode

#33

Post by politelemon »

james2doyle wrote: post #12
thewataccount wrote: post #11

I can't seem to find a demo, if anyone has a chance to test it, how does it compare to replit and wizardcoder?

Looks like there is one on the Hugging Face page: https://huggingface.co/stabilityai/stablecode-instruct-alpha... Not very promising based on this lame test

I ran it locally and it seemed to do better. I switched Python to Bash and it also gave a good answer (nproc).

Re: StableCode

#34

Post by cutler »

Yet another site whose data privacy policy amounts to nothing more than an Accept button. Refuse to use such sites.

Re: StableCode

#35

Post by lolinder »

miohtama wrote: post #31
3rd3 wrote: post #26

How does it compare to GitHub Copilot?

The model, source, etc. are available under permissive terms https://huggingface.co/stabilityai/stablecode-instruct-alpha... You can “run it locally”. Very handy if you do not trust automatically sending all your code to someone in the United States.

> to reproduce, distribute, and create derivative works of the Software Products solely for your non-commercial research purposes

I wouldn't call these terms permissive. It's in line with the recent trend in released AI models, but fairly restrictive in what you're actually allowed to do with it.

Re: StableCode

#36

Post by _pdp_ »

rvz wrote: post #3

Either way, the race to zero has been further accelerated. Stability AI, Apple, Meta, etc are clearly at the finish line putting pressure on cloud only AI models and cannot raise prices or compete with free.

Lots of folks out there would rather skip the hassle of running their own models, and that's totally understandable. Similarly, you've got plenty of folks who'd rather pay for managed hosting services instead of dealing with the nitty-gritty of setting up everything themselves using free tools. This opens up exciting opportunities for successful companies to offer some real perks – think convenience, a smoother user experience, and lightning-fast speeds, just to name a few! All of these things save time and are worth paying for.

Re: StableCode

#37

Post by sebzim4500 »

jstummbillig wrote: post #30
3rd3 wrote: post #26

How does it compare to GitHub Copilot?

When they don't voluntarily answer the question, you know the answer.

It's not easy to compare them, to be fair.

I guess you could come up with a thousand example prompts and pay some students to pick which output is better, but I can also see why you wouldn't bother. It probably depends on language, type of prompt, etc.

Re: StableCode

#38

Post by capableweb »

cutler wrote: post #34

Yet another site whose data privacy policy amounts to nothing more than an Accept button. Refuse to use such sites.

It's a model you download and run yourself, on your own hardware. No privacy policy needed.

Re: StableCode

#39

Post by erwald »

sebzim4500 wrote: post #37
jstummbillig wrote: post #30

Earlier quoted context omitted.

When they don't voluntarily answer the question, you know the answer.

It's not easy to compare them, to be fair. I guess you could come up with a thousand example prompts and pay some students to pick which output is better, but I can also see why you wouldn't bother. It probably depends on language, type of prompt, etc.

Sure it's easy -- you can use benchmarks like HumanEval, which Stability did. They just didn't compare to Codex or GPT-4. Of course such benchmarks don't capture all aspects of an LLM's capabilities, but they're a lot better than nothing!

Re: StableCode

#40

Post by capableweb »

sebzim4500 wrote: post #2

Hard to believe it can work that well when it only has 3B parameters, but I'd love to be proven wrong.

I had that thought at first too, but then the scope is really small (programming) compared to other models (everything) so might not be that bad.
Post reply on HN