Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Stable Code 3B: Coding on the Edge

stability.ai

Post reply on HN
Print view View original

31–40 of 148 posts

Re: Stable Code 3B: Coding on the Edge

#31

Post by yorwba »

kleiba wrote: post #14

It's quite amazing - I often find that I read quite positive comments towards LLM tools for coding. Yet, an "Ask HN" I posted a while ago (and which admittedly didn't gain much traction) seemed to mirror mostly negative/pessimistic responses. https://news.ycombinator.com/item?id=38803836 Was it just that my submission didn't find enough / more balanced commenters?

You got two positive and two negative responses. You replied only to the negative responses. Now you think that the responses were mostly negative. I blame salience bias.

Anyways, there's also a difference between "are you excited about this new thing becoming available" and "now that you've used it, do you like the experience". The former is more likely to feature rosy expectations and the latter bitter disappointment. (Though it could also be the other way around, with people dismissing it at first and then discovering that it's kind of nice actually.)

Re: Stable Code 3B: Coding on the Edge

#32

Post by tarruda »

knicholes wrote: post #9

I've got a machine with 4 3090s-- Anyone know which model would perform the best for programming? It's great this can run on a machine w/out a graphics card and is only 3B params, but I have the hardware. Might as well use it.

AFAIK deepseek coder family are the best open coding models.

I haven't tested, but I think deepseek coder 33b can run in a single RTX 3090 when 4-bit quantized. In your case you might be able to run the non quantized version

Re: Stable Code 3B: Coding on the Edge

#33

Post by Havoc »

kleiba wrote: post #14

It's quite amazing - I often find that I read quite positive comments towards LLM tools for coding. Yet, an "Ask HN" I posted a while ago (and which admittedly didn't gain much traction) seemed to mirror mostly negative/pessimistic responses. https://news.ycombinator.com/item?id=38803836 Was it just that my submission didn't find enough / more balanced commenters?

The precise wording matters.

How has it changed your work life leads people down the rabbit hole of will coding jobs be safe.

This one is a lot more neutral/technical.

Re: Stable Code 3B: Coding on the Edge

#34

Post by Havoc »

knicholes wrote: post #9

I've got a machine with 4 3090s-- Anyone know which model would perform the best for programming? It's great this can run on a machine w/out a graphics card and is only 3B params, but I have the hardware. Might as well use it.

The coding models are all small because speed is crucial. If you need to wait 2 seconds for an autocomplete it becomes near useless.

Re: Stable Code 3B: Coding on the Edge

#35

Post by danielbln »

tarruda wrote: post #10

Note that they don't compare with deepseek coder 6.7b, which is vastly superior to much bigger coding models. Surpassing codellama 7b is not that big of a deal today. The most impressive thing about these results is how good the 1.3B deepseek coder is.

Deepseek-coder-6.7B really is a quite surprisingly capable model. It's easy to give it a spin with ollama via `ollama run deepseek-coder:6.7b`.

Re: Stable Code 3B: Coding on the Edge

#36

Post by debarshri »

joshmarlow wrote: post #17
keyle wrote: post #2

That is fantastic. I'm building a small macOS SwiftUI client with llama cpp built in, no server-client model, and it's already so useful with models like openhermes chat 7B, and fast. If this opens it to smaller laptops, wow! We truly live in crazy time. The rate of improvement in this field is off the walls.

Not sure if this is where your head is, but I think there's a lot of value in integrating LLMs directly into complex software. Jira, Salesforce, maybe K8s - should all have an integrated LLMs that can walk you through how to perform a nuanced task in the software.

Walkthrough is generally performed once or not so frequently. It would be a bad investment if you just use it for just this use case

Re: Stable Code 3B: Coding on the Edge

#37

Post by SparkyMcUnicorn »

SushiHippie wrote: post #18
knicholes wrote: post #9

I've got a machine with 4 3090s-- Anyone know which model would perform the best for programming? It's great this can run on a machine w/out a graphics card and is only 3B params, but I have the hardware. Might as well use it.

Here is a leader board of some models https://huggingface.co/spaces/mike-ravkine/can-ai-code-resul... Don't know how biased this leaderboard is, but I guess you could just give some of them a try and see for yourself.

This is a much better leaderboard: https://evalplus.github.io/leaderboard.html

I've seen the CanAiCode leaderboard several times (and used many of the models listed), but I wouldn't use it to pick a model. It's not a bad list, but the benchmark is too limited. The results are not accurately ranked from best to worst.

For example the deepseek 33b model is ranked 5 spots lower than the 6.7b model, but the 33b model is definitely better. WizardCoder 15b is near the top while WizardCoder 33b is ranked 26 spots lower, which is a wildly inaccurate ranking.

It's worth noting that those 33b models score in the 70s for HumanEval and HumanEval+ while the 15b model scores in the 50s.

Re: Stable Code 3B: Coding on the Edge

#39

Post by alpaca128 »

debarshri wrote: post #36
joshmarlow wrote: post #17

Earlier quoted context omitted.

Not sure if this is where your head is, but I think there's a lot of value in integrating LLMs directly into complex software. Jira, Salesforce, maybe K8s - should all have an integrated LLMs that can walk you through how to perform a nuanced task in the software.

Walkthrough is generally performed once or not so frequently. It would be a bad investment if you just use it for just this use case

A beginner tutorial is also not used frequently by users, but that doesn't make it a bad investment. I an LLM can help a lot with getting familiar with the tool it could be pretty valuable, especially after a UI rework etc.

Re: Stable Code 3B: Coding on the Edge

#40

Post by triyambakam »

danielbln wrote: post #35
tarruda wrote: post #10

Note that they don't compare with deepseek coder 6.7b, which is vastly superior to much bigger coding models. Surpassing codellama 7b is not that big of a deal today. The most impressive thing about these results is how good the 1.3B deepseek coder is.

Deepseek-coder-6.7B really is a quite surprisingly capable model. It's easy to give it a spin with ollama via `ollama run deepseek-coder:6.7b`.

Thanks for the tip with ollama
Post reply on HN