Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Stable Code 3B: Coding on the Edge

stability.ai

Post reply on HN
Print view View original

81–90 of 148 posts

Re: Stable Code 3B: Coding on the Edge

#81

Post by emadm »

sangnoir wrote: post #55
herval wrote: post #52

Can anyone explain what’s Stability’s business model (or plan for one)? I get why Meta releases tons of models, but still can’t quite understand what stability is trying to achieve

Seems like the standard open-core playbook: > This model is included in our new Stability AI Membership. Visit our Membership page to take advantage of our commercial Core Model offerings, including SDXL Turbo & Stable Video Diffusion. A hypothetical Stable Code 13B/70B could be hosted only, with more languages or specialized use-cases (Stable Code 3B iOS-Swift-Turbo)

Membership with upsell to support, custom models and more

Plus licensed variant models like stable audio and on chip installation like arm for specialist models eg Japanese law or Indonesian accounting

Re: Stable Code 3B: Coding on the Edge

#82

Post by refulgentis »

emadm wrote: post #77
JCM9 wrote: post #56

Don’t entirely understand Stability’s business model. They’ve been putting out a lot of models recently and Stable Diffusion was novel at the time, but now their models consistently seem to be somewhat second rate compared to other things out there. For example Midjourney now seems to have far surpassed them in the image generation front. After raising a ton of funding Stability seems to just be throwing a bunch of s…

Business model is bundling so you have a one stop shop for good quality models of every modality and cultural variants of them. These go on bedrock, on chip, on prem etc and our consulting partners take them to the end user. On the innovation side stable diffusion turbo does like 100 cats with hats per second and the video model outperforms runway, pika etc on blind tests. Stable audio was one of the time innovation…

Vouch, I finally tried Stable LM 3b zephyr today and I'm stunned this slipped by. It's the only model I've tried that's not Mistral 7B that can do RAG. And it can run ~any consumer grade hardware released in last 3 years. I'm literally stunned it's been sitting out since December 8th. I've heard 10x more about Phi-2 than it, and I'm not sure why.

(Official ONNX version, please!! Then you get Transformers.js / web / I can deploy on every platform from Web to iOS to Windows)

re: art, Dalle-3 costs significantly more. XL costs are 1/5th of what they were at launch, 0.0002/image versus Dalle-3's 0.04. And you'd be surprised how often people are happy with XL -- Dalle-3's marginal advantage is mostly text, especially with the excessive filtering of stylistic stuff, and forced prompt rewrites

Re: Stable Code 3B: Coding on the Edge

#83

Post by wokwokwok »

a_wild_dandan wrote: post #63
jyap wrote: post #50

Earlier quoted context omitted.

Deepseek Coder Instruct 6.7b has been my local LLM (M1 series MBP) for a while now and that was my first thought… They selectively chose benchmark results to look impressive (which is typical). I tested out StableLM Zephyr 3B when that came out and it was extremely underwhelming/unusable. Based on this, Stable Code 3B doesn’t look to be worth trying out. Guessing if they could put out a 7B model which beat Deepseek C…

Do you know how Deepseek 33b compares to 6.7b? I'm trying 33b on my (96GB) MacBook just because I have plenty of spare (V)RAM. But I'll run the smaller model if the benefits are marginal in other peoples' experience.

The smaller model is great at trivial day-to-day tasks.

However, when you ask hard things, it struggles; you can ask the same question 10 times, and only get 1 answer that actually answers the question.

...but the larger model is a lot slower.

Generally, if you don't want to mess around swapping models, stick with the bigger one. It's better.

However, if you are heavily using it, you'll find the speed is a pain in the ass, and when you want a trivial hint like 'how do I do a map statement in kotlin again?', you really don't need it.

What I have setup personally is a little thumbs-up / thumbs-down on the suggestions via a custom intellij plugin; if I 'thumbs-down' a result, it generates a new solution for it.

If I 'thumbs-down' it twice, it swaps to the larger model to generate a solution for it.

This kind of 'use ok model for most things and step up to larger model when you start asking hard stuff' approach scales very nicely for my personal workflow... but, I admit that setting it up was a pain, and I'm forever pissing around with the plugin code to fix tiny bugs, which I would prefer to be spending doing actual work.

So... there's not really much tooling out there at the moment to support it, but the best solution really is to use both.

If you don't want to and just want 'use the best model for everything', stick with the bigger one.

The larger model is more capable of turning 'here is a description of what I want' into 'here is code that does it that actually compiles'.

The smaller model is much better at 'I want a code fragment that does X' -> 'rephrased stack overflow answer'.

Re: Stable Code 3B: Coding on the Edge

#84

Post by jyap »

sroussey wrote: post #72
jyap wrote: post #62

Earlier quoted context omitted.

Golang grinding Leetcode coding buddy. And general coding buddy PR reviewer. The results are on par or better than ChatGPT 3.5. I often use it to delve deeper such as “is there an alternative way to write this?” Or “how does this code look?” If you have an M-series Mac I recommend trying out LM Studio. Really eye opening and I’m excited to see how things progress.

I have GitHub copilot. Is it better than that? Nd if so, in which way? Offline would be one for sure. Cost is another. What else?

I’ve never used GH Copilot so can’t comment on that.

But having everything locally means no privacy or data leak issues.

Re: Stable Code 3B: Coding on the Edge

#85

Post by SubiculumCode »

jyap wrote: post #50
tarruda wrote: post #10

Note that they don't compare with deepseek coder 6.7b, which is vastly superior to much bigger coding models. Surpassing codellama 7b is not that big of a deal today. The most impressive thing about these results is how good the 1.3B deepseek coder is.

Deepseek Coder Instruct 6.7b has been my local LLM (M1 series MBP) for a while now and that was my first thought… They selectively chose benchmark results to look impressive (which is typical). I tested out StableLM Zephyr 3B when that came out and it was extremely underwhelming/unusable. Based on this, Stable Code 3B doesn’t look to be worth trying out. Guessing if they could put out a 7B model which beat Deepseek C…

How do you make use of it? Do you have it integrated directly into an ide?

Re: Stable Code 3B: Coding on the Edge

#87

Post by hskalin »

eyegor wrote: post #76
tarruda wrote: post #10

Note that they don't compare with deepseek coder 6.7b, which is vastly superior to much bigger coding models. Surpassing codellama 7b is not that big of a deal today. The most impressive thing about these results is how good the 1.3B deepseek coder is.

The 1.3b model is amazing for real time code complete, it's fast enough to be a better intellisense. Another model you should try is magicoder 6.7b ds (based on deepseek coder). After playing with it for a couple weeks, I think it gives slightly better results than the equivalent deepseek model. Repo https://github.com/ise-uiuc/magicoder Models https://huggingface.co/models?search=Magicoder-s-ds

How do you use these models with your editor? (E. vscode or Emacs etc)

Re: Stable Code 3B: Coding on the Edge

#88

Post by satvikpendem »

JCM9 wrote: post #56

Don’t entirely understand Stability’s business model. They’ve been putting out a lot of models recently and Stable Diffusion was novel at the time, but now their models consistently seem to be somewhat second rate compared to other things out there. For example Midjourney now seems to have far surpassed them in the image generation front. After raising a ton of funding Stability seems to just be throwing a bunch of s…

> For example Midjourney now seems to have far surpassed them in the image generation front

Nope. Stable Diffusion with alternative models offers far more customization and control than Midjourney. Midjourney is good for beginners but sucks for experts.

Re: Stable Code 3B: Coding on the Edge

#89

Post by mlboss »

knicholes wrote: post #9

I've got a machine with 4 3090s-- Anyone know which model would perform the best for programming? It's great this can run on a machine w/out a graphics card and is only 3B params, but I have the hardware. Might as well use it.

Did you build a machine with 4x 3090 ? I looking for a way to build such a machine for ML training.

Re: Stable Code 3B: Coding on the Edge

#90

Post by tarruda »

wokwokwok wrote: post #83
a_wild_dandan wrote: post #63

Earlier quoted context omitted.

Do you know how Deepseek 33b compares to 6.7b? I'm trying 33b on my (96GB) MacBook just because I have plenty of spare (V)RAM. But I'll run the smaller model if the benefits are marginal in other peoples' experience.

The smaller model is great at trivial day-to-day tasks. However, when you ask hard things, it struggles; you can ask the same question 10 times, and only get 1 answer that actually answers the question. ...but the larger model is a lot slower. Generally, if you don't want to mess around swapping models, stick with the bigger one. It's better. However, if you are heavily using it, you'll find the speed is a pain in th…

> but the larger model is a lot slower.

I found the performance to be very acceptable for 33b 4 bit on a m3 max with 36gb ram (much faster than reading speed)

Post reply on HN