Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Stable Code 3B: Coding on the Edge

stability.ai

Post reply on HN
Print view View original

91–100 of 148 posts

Re: Stable Code 3B: Coding on the Edge

#91

Post by tarruda »

hskalin wrote: post #87
eyegor wrote: post #76

Earlier quoted context omitted.

The 1.3b model is amazing for real time code complete, it's fast enough to be a better intellisense. Another model you should try is magicoder 6.7b ds (based on deepseek coder). After playing with it for a couple weeks, I think it gives slightly better results than the equivalent deepseek model. Repo https://github.com/ise-uiuc/magicoder Models https://huggingface.co/models?search=Magicoder-s-ds

How do you use these models with your editor? (E. vscode or Emacs etc)

An easy way is to use an OpenAI compatible server, which means you can use any GPT plugin to integrate with your editor

Re: Stable Code 3B: Coding on the Edge

#92

Post by WatchDog »

devindotcom wrote: post #49
jjtheblunt wrote: post #46

Jargon naivete question: isn't "on the edge" normally implying on a server side with minimal routers hops to the client, not on client side?

afaik "edge" nearly always means taking place on the device a user is interacting with. no server involved except perhaps as authentication etc. but there is probably some other situation where "edge" could mean local infra or caching.

I think the etymology of “edge computing” is derived from “network edge”, ie the outer shell of some network/autonomous system.

The closest point within your control that interfaces with devices outside of your control.

Seeing the term get used to describe client devices themselves kinda muddies the terminology.

Re: Stable Code 3B: Coding on the Edge

#93

Post by unshavedyak »

a_wild_dandan wrote: post #44
tarruda wrote: post #10

Note that they don't compare with deepseek coder 6.7b, which is vastly superior to much bigger coding models. Surpassing codellama 7b is not that big of a deal today. The most impressive thing about these results is how good the 1.3B deepseek coder is.

This is phenomenal. And runs fast! The 33b version might be my MacBook's new coding daily driver.

How are you using it? I need to find some sane way to use this stuff from Helix/terminal..

Re: Stable Code 3B: Coding on the Edge

#94

Post by alwinaugustin »

I've been experimenting with code-llama extensively on my laptop, and from my experience, it seems that these models are still in their early stages. I primarily utilize them through a Web UI, where they can successfully refactor code given an existing snippet. However, it's worth noting that they cannot currently analyze entire codebases or packages, refining them based on the most suitable solutions using the most appropriate algorithms. While these models offer assistance to some extent, there is room for improvement in their ability to handle more complex and comprehensive coding scenarios.

Re: Stable Code 3B: Coding on the Edge

#95

Post by danielmarkbruce »

alwinaugustin wrote: post #94

I've been experimenting with code-llama extensively on my laptop, and from my experience, it seems that these models are still in their early stages. I primarily utilize them through a Web UI, where they can successfully refactor code given an existing snippet. However, it's worth noting that they cannot currently analyze entire codebases or packages, refining them based on the most suitable solutions using the most…

I think there is a decent chance SourceGraph will figure this all out. The most important thing at this point is figuring what context to feed. They can build up a nice graph of a codebase and I expect from there they can put in the best context and then boom.

They might also be able to train a model more intelligently by generating training data from said graphs.

Re: Stable Code 3B: Coding on the Edge

#96

Post by wokwokwok »

tarruda wrote: post #90
wokwokwok wrote: post #83

Earlier quoted context omitted.

The smaller model is great at trivial day-to-day tasks. However, when you ask hard things, it struggles; you can ask the same question 10 times, and only get 1 answer that actually answers the question. ...but the larger model is a lot slower. Generally, if you don't want to mess around swapping models, stick with the bigger one. It's better. However, if you are heavily using it, you'll find the speed is a pain in th…

> but the larger model is a lot slower. I found the performance to be very acceptable for 33b 4 bit on a m3 max with 36gb ram (much faster than reading speed)

I’m not sure what to say; responsive fast output is ideal, and the larger model is distinctly slower for me, particularly for long completions (2k tokens) if you’re using a restricted grammar like json output.

I’m using an M2 not an M3 though; maybe it’s better for you.

I was under the impression quantised results were generally slower too, but I’ve never dug into it (or particularly noticed a difference between q4/q5/q6).

If you find it fast enough to use then go for it~

Re: Stable Code 3B: Coding on the Edge

#97

Post by liuliu »

JCM9 wrote: post #65
liuliu wrote: post #61

Earlier quoted context omitted.

> one can just quickly swap it out for something else someone else paid to train. That doesn't seem to be the case. There are very limited open-source models outside of the small-LLM bubble.

The open space on small models is a whole other developing angle, but O was referring to the general commoditization of a lot of these models. With rare exception after launch it seems the lifespan of any of these models is rather limited. From a business standpoint that sort of scenario is generally very unattractive and thus was trying to understand if they have some other angle they’re trying to play here to make…

The comoditization of models outside of LLM is delusional. There is no comparable open source image / video models to the private ones.

Re: Stable Code 3B: Coding on the Edge

#98

Post by doctorpangloss »

emadm wrote: post #77
JCM9 wrote: post #56

Don’t entirely understand Stability’s business model. They’ve been putting out a lot of models recently and Stable Diffusion was novel at the time, but now their models consistently seem to be somewhat second rate compared to other things out there. For example Midjourney now seems to have far surpassed them in the image generation front. After raising a ton of funding Stability seems to just be throwing a bunch of s…

Business model is bundling so you have a one stop shop for good quality models of every modality and cultural variants of them. These go on bedrock, on chip, on prem etc and our consulting partners take them to the end user. On the innovation side stable diffusion turbo does like 100 cats with hats per second and the video model outperforms runway, pika etc on blind tests. Stable audio was one of the time innovation…

I use Stable Diffusion family models for innovative art products.

On a small scale, you have to professionalize ComfyUI’s development. My PR to make it installable and to make a plugin ecosystem that makes sense should not be sitting unmerged (https://github.com/comfyanonymous/ComfyUI/pull/298).

On a medium scale, CLIP is holding you back. I would eagerly buy a 48GB card to accommodate a batch size 1, gradient checkpointed LoRA-trainable model with T5 for conditioning. I want PixArt-a or DeepFloyd/IF with the SDXL dataset and training. I get I can achieve so much with SDXL on 24GB, including just barely a fine tuning, I understand the engineering decisions here, but it’s too weak on prompts.

On a large scale, I’m willing to spend a little money up front. In those conditions you can be far more innovative, you don’t have to make everything for $0. Shane Carruth didn’t make Primer for $0. I’m sure you’ve seen this movie, you get how astoundingly good it is. But he still spent something. He spent only slightly more than an RTX 6000 Ada.

Innovators have budgets. It’s still worth releasing the most powerful possible model for expensive hardware, this is why everyone is talking about Mixtral, but it’s especially true of visual art.

Re: Stable Code 3B: Coding on the Edge

#99

Post by behnamoh »

triyambakam wrote: post #40
danielbln wrote: post #35

Earlier quoted context omitted.

Deepseek-coder-6.7B really is a quite surprisingly capable model. It's easy to give it a spin with ollama via `ollama run deepseek-coder:6.7b`.

Thanks for the tip with ollama

ollama is actually not a great way to run these models as it makes it difficult to change server parameters and doesn't use `mlock` to keep the models in memory.

Re: Stable Code 3B: Coding on the Edge

#100

Post by jjtheblunt »

WatchDog wrote: post #92
devindotcom wrote: post #49

Earlier quoted context omitted.

afaik "edge" nearly always means taking place on the device a user is interacting with. no server involved except perhaps as authentication etc. but there is probably some other situation where "edge" could mean local infra or caching.

I think the etymology of “edge computing” is derived from “network edge”, ie the outer shell of some network/autonomous system. The closest point within your control that interfaces with devices outside of your control. Seeing the term get used to describe client devices themselves kinda muddies the terminology.

Agreed.

(autocorrect style typo above: etymology)

Post reply on HN