Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Stable Code 3B: Coding on the Edge

stability.ai

Post reply on HN
Print view View original

61–70 of 148 posts

Re: Stable Code 3B: Coding on the Edge

#61

Post by liuliu »

JCM9 wrote: post #56

Don’t entirely understand Stability’s business model. They’ve been putting out a lot of models recently and Stable Diffusion was novel at the time, but now their models consistently seem to be somewhat second rate compared to other things out there. For example Midjourney now seems to have far surpassed them in the image generation front. After raising a ton of funding Stability seems to just be throwing a bunch of s…

> one can just quickly swap it out for something else someone else paid to train.

That doesn't seem to be the case. There are very limited open-source models outside of the small-LLM bubble.

Re: Stable Code 3B: Coding on the Edge

#62

Post by jyap »

sroussey wrote: post #58
jyap wrote: post #50

Earlier quoted context omitted.

Deepseek Coder Instruct 6.7b has been my local LLM (M1 series MBP) for a while now and that was my first thought… They selectively chose benchmark results to look impressive (which is typical). I tested out StableLM Zephyr 3B when that came out and it was extremely underwhelming/unusable. Based on this, Stable Code 3B doesn’t look to be worth trying out. Guessing if they could put out a 7B model which beat Deepseek C…

What do you use it for?

Golang grinding Leetcode coding buddy. And general coding buddy PR reviewer.

The results are on par or better than ChatGPT 3.5.

I often use it to delve deeper such as “is there an alternative way to write this?” Or “how does this code look?”

If you have an M-series Mac I recommend trying out LM Studio. Really eye opening and I’m excited to see how things progress.

Re: Stable Code 3B: Coding on the Edge

#63

Post by a_wild_dandan »

jyap wrote: post #50
tarruda wrote: post #10

Note that they don't compare with deepseek coder 6.7b, which is vastly superior to much bigger coding models. Surpassing codellama 7b is not that big of a deal today. The most impressive thing about these results is how good the 1.3B deepseek coder is.

Deepseek Coder Instruct 6.7b has been my local LLM (M1 series MBP) for a while now and that was my first thought… They selectively chose benchmark results to look impressive (which is typical). I tested out StableLM Zephyr 3B when that came out and it was extremely underwhelming/unusable. Based on this, Stable Code 3B doesn’t look to be worth trying out. Guessing if they could put out a 7B model which beat Deepseek C…

Do you know how Deepseek 33b compares to 6.7b? I'm trying 33b on my (96GB) MacBook just because I have plenty of spare (V)RAM. But I'll run the smaller model if the benefits are marginal in other peoples' experience.

Re: Stable Code 3B: Coding on the Edge

#64

Post by bogwog »

seydor wrote: post #54
herval wrote: post #52

Can anyone explain what’s Stability’s business model (or plan for one)? I get why Meta releases tons of models, but still can’t quite understand what stability is trying to achieve

to be bought by meta

This is all an elaborate mating ritual

Re: Stable Code 3B: Coding on the Edge

#65

Post by JCM9 »

liuliu wrote: post #61
JCM9 wrote: post #56

Don’t entirely understand Stability’s business model. They’ve been putting out a lot of models recently and Stable Diffusion was novel at the time, but now their models consistently seem to be somewhat second rate compared to other things out there. For example Midjourney now seems to have far surpassed them in the image generation front. After raising a ton of funding Stability seems to just be throwing a bunch of s…

> one can just quickly swap it out for something else someone else paid to train. That doesn't seem to be the case. There are very limited open-source models outside of the small-LLM bubble.

The open space on small models is a whole other developing angle, but O was referring to the general commoditization of a lot of these models. With rare exception after launch it seems the lifespan of any of these models is rather limited. From a business standpoint that sort of scenario is generally very unattractive and thus was trying to understand if they have some other angle they’re trying to play here to make a viable business out of this. Or the business model can just be get acquired before that matters and let that be someone else’s problem to figure out.

Re: Stable Code 3B: Coding on the Edge

#67

Post by smusamashah »

JCM9 wrote: post #56

Don’t entirely understand Stability’s business model. They’ve been putting out a lot of models recently and Stable Diffusion was novel at the time, but now their models consistently seem to be somewhat second rate compared to other things out there. For example Midjourney now seems to have far surpassed them in the image generation front. After raising a ton of funding Stability seems to just be throwing a bunch of s…

Midjourney has better quality but does not offer any control. Community has done and is still doing a a lot with SD models because they can be played and tinkered with in any way anyone wants to.

Re: Stable Code 3B: Coding on the Edge

#68

Post by [deleted] »

JCM9 wrote: post #56

Don’t entirely understand Stability’s business model. They’ve been putting out a lot of models recently and Stable Diffusion was novel at the time, but now their models consistently seem to be somewhat second rate compared to other things out there. For example Midjourney now seems to have far surpassed them in the image generation front. After raising a ton of funding Stability seems to just be throwing a bunch of s…

[deleted]

Re: Stable Code 3B: Coding on the Edge

#69

Post by washadjeffmad »

JCM9 wrote: post #56

Don’t entirely understand Stability’s business model. They’ve been putting out a lot of models recently and Stable Diffusion was novel at the time, but now their models consistently seem to be somewhat second rate compared to other things out there. For example Midjourney now seems to have far surpassed them in the image generation front. After raising a ton of funding Stability seems to just be throwing a bunch of s…

Midjourney is decidedly underwhelming if you've spent any time using the expansive tooling and control nets of Stable Diffusion. Yes, it's easy to get impressive first gens with MJ, but all of the coolest work and integration happening is using SD.

Re: Stable Code 3B: Coding on the Edge

#70

Post by SushiHippie »

SparkyMcUnicorn wrote: post #37
SushiHippie wrote: post #18

Earlier quoted context omitted.

Here is a leader board of some models https://huggingface.co/spaces/mike-ravkine/can-ai-code-resul... Don't know how biased this leaderboard is, but I guess you could just give some of them a try and see for yourself.

This is a much better leaderboard: https://evalplus.github.io/leaderboard.html I've seen the CanAiCode leaderboard several times (and used many of the models listed), but I wouldn't use it to pick a model. It's not a bad list, but the benchmark is too limited. The results are not accurately ranked from best to worst. For example the deepseek 33b model is ranked 5 spots lower than the 6.7b model, but the 33b model is…

Thanks
Post reply on HN