Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Stable Code 3B: Coding on the Edge

stability.ai

Post reply on HN
Print view View original

71–80 of 148 posts

Re: Stable Code 3B: Coding on the Edge

#71

Post by jyap »

a_wild_dandan wrote: post #63
jyap wrote: post #50

Earlier quoted context omitted.

Deepseek Coder Instruct 6.7b has been my local LLM (M1 series MBP) for a while now and that was my first thought… They selectively chose benchmark results to look impressive (which is typical). I tested out StableLM Zephyr 3B when that came out and it was extremely underwhelming/unusable. Based on this, Stable Code 3B doesn’t look to be worth trying out. Guessing if they could put out a 7B model which beat Deepseek C…

Do you know how Deepseek 33b compares to 6.7b? I'm trying 33b on my (96GB) MacBook just because I have plenty of spare (V)RAM. But I'll run the smaller model if the benefits are marginal in other peoples' experience.

You would want to test it out manually day to day. That’s always the best. Some models can out score but not actually be “better” when you use it.

But there is also the benchmarking: https://github.com/deepseek-ai/deepseek-coder

33B Instruct doesn’t beat 6.7B Instruct by much but maybe those % improvements mean more for your usage.

I run 6.7B since I have 16GB RAM.

Quantization of the model also makes a difference.

Re: Stable Code 3B: Coding on the Edge

#72

Post by sroussey »

jyap wrote: post #62
sroussey wrote: post #58

Earlier quoted context omitted.

What do you use it for?

Golang grinding Leetcode coding buddy. And general coding buddy PR reviewer. The results are on par or better than ChatGPT 3.5. I often use it to delve deeper such as “is there an alternative way to write this?” Or “how does this code look?” If you have an M-series Mac I recommend trying out LM Studio. Really eye opening and I’m excited to see how things progress.

I have GitHub copilot. Is it better than that? Nd if so, in which way?

Offline would be one for sure. Cost is another. What else?

Re: Stable Code 3B: Coding on the Edge

#76

Post by eyegor »

tarruda wrote: post #10

Note that they don't compare with deepseek coder 6.7b, which is vastly superior to much bigger coding models. Surpassing codellama 7b is not that big of a deal today. The most impressive thing about these results is how good the 1.3B deepseek coder is.

The 1.3b model is amazing for real time code complete, it's fast enough to be a better intellisense.

Another model you should try is magicoder 6.7b ds (based on deepseek coder). After playing with it for a couple weeks, I think it gives slightly better results than the equivalent deepseek model.

Repo https://github.com/ise-uiuc/magicoder

Models https://huggingface.co/models?search=Magicoder-s-ds

Re: Stable Code 3B: Coding on the Edge

#77

Post by emadm »

JCM9 wrote: post #56

Don’t entirely understand Stability’s business model. They’ve been putting out a lot of models recently and Stable Diffusion was novel at the time, but now their models consistently seem to be somewhat second rate compared to other things out there. For example Midjourney now seems to have far surpassed them in the image generation front. After raising a ton of funding Stability seems to just be throwing a bunch of s…

Business model is bundling so you have a one stop shop for good quality models of every modality and cultural variants of them.

These go on bedrock, on chip, on prem etc and our consulting partners take them to the end user.

On the innovation side stable diffusion turbo does like 100 cats with hats per second and the video model outperforms runway, pika etc on blind tests.

Stable audio was one of the time innovation of the year winners on music and we released a sota 3d model.

Stable LM zephyr is the best 3b chat model works great on a MacBook Air.

Most of the pixels in the world will be generated so fast high quality image/video are the core and these other models are to support them.

It’s really hard to build good solid models and we are the only company that can build a model of any type for anyone.

Re: Stable Code 3B: Coding on the Edge

#78

Post by mistercheph »

knicholes wrote: post #9

I've got a machine with 4 3090s-- Anyone know which model would perform the best for programming? It's great this can run on a machine w/out a graphics card and is only 3B params, but I have the hardware. Might as well use it.

Try mistral 8x7b, which some human evals place above gpt-3.5 and you have enough VRAM and compute to make training a LORA either on your own dataset, or one of the freely available datasets on huggingface worthwhile, or at least interesting

Re: Stable Code 3B: Coding on the Edge

#79

Post by emadm »

keyle wrote: post #2

That is fantastic. I'm building a small macOS SwiftUI client with llama cpp built in, no server-client model, and it's already so useful with models like openhermes chat 7B, and fast. If this opens it to smaller laptops, wow! We truly live in crazy time. The rate of improvement in this field is off the walls.

3b is good for 8gb MacBook Air etc. 7b is slightly too big.

Sure these will continue to improve, phi2 is a good base as well

Re: Stable Code 3B: Coding on the Edge

#80

Post by emadm »

artninja1988 wrote: post #4

Given the complete failure of the first stable lm, I'm interested to try this one out. Haven't really seen a small language model, except mixtral 7b that's really useful for much. I also hope stability comes out with a competitor to the new midjourney and dalle models! That's what put them on the map in the first place

We released a competitor to runway recently that beat it on blind tests, plus way faster image in sdxl turbo

We have been working on ComfyUI for the next step and new image models

Midjourney and others are pipelines versus models so we have a higher bar to jump but the og stable diffusion team are working hard!

Post reply on HN