How is this compared to the current GitHub Copilot?
A 3B tiny model is not going to compare to GitHub copilot. However, there are plenty of nice 7B models that are excellent at code and I encourage you to try them out.
Stable Code 3B: Coding on the Edge
11–20 of 148 posts
Re: Stable Code 3B: Coding on the Edge
#12FYI: This model is already available on Ollama.
Re: Stable Code 3B: Coding on the Edge
#13Given the complete failure of the first stable lm, I'm interested to try this one out. Haven't really seen a small language model, except mixtral 7b that's really useful for much. I also hope stability comes out with a competitor to the new midjourney and dalle models! That's what put them on the map in the first place
All of the Mistral versions have been excellent, including the OpenHermes versions. I encourage you to check out Phi-2 as well, it's the only 3b model I've found really quite interesting outside of Replit's code model built into Replit Core.
Re: Stable Code 3B: Coding on the Edge
#14It's quite amazing - I often find that I read quite positive comments towards LLM tools for coding. Yet, an "Ask HN" I posted a while ago (and which admittedly didn't gain much traction) seemed to mirror mostly negative/pessimistic responses.
https://news.ycombinator.com/item?id=38803836
Was it just that my submission didn't find enough / more balanced commenters?
Re: Stable Code 3B: Coding on the Edge
#15How reliable are these benchmarks?
I think the trick is that they are just comparing to other tiny models.
None of the little models, including this one, are comparable to the performance of the larger models for any significant coding problem.
I think what these are useful for is mostly giving people hints inside of a code editor. Occasionally filling in the blank.
Re: Stable Code 3B: Coding on the Edge
#16I just tried this model with Koboldcpp on my LLM box. I got gibberish back.
My prompt - "please show me how to write a web scraper in Python"
The response?
I've written my first ever python script about 5 months ago and I really don't remember anything except for the fact that I used Selenium in order to scrape websites (in this case, Google). So you can probably just copy/paste all of these lines from your own Python code which contains logic to determine what value should be returned when called by another piece of software or program.
Re: Stable Code 3B: Coding on the Edge
#17That is fantastic. I'm building a small macOS SwiftUI client with llama cpp built in, no server-client model, and it's already so useful with models like openhermes chat 7B, and fast. If this opens it to smaller laptops, wow! We truly live in crazy time. The rate of improvement in this field is off the walls.
Not sure if this is where your head is, but I think there's a lot of value in integrating LLMs directly into complex software. Jira, Salesforce, maybe K8s - should all have an integrated LLMs that can walk you through how to perform a nuanced task in the software.
Re: Stable Code 3B: Coding on the Edge
#18I've got a machine with 4 3090s-- Anyone know which model would perform the best for programming? It's great this can run on a machine w/out a graphics card and is only 3B params, but I have the hardware. Might as well use it.
Here is a leader board of some models
https://huggingface.co/spaces/mike-ravkine/can-ai-code-resul...
Don't know how biased this leaderboard is, but I guess you could just give some of them a try and see for yourself.
Re: Stable Code 3B: Coding on the Edge
#19How is this compared to the current GitHub Copilot?
If you just want to get stuff done, use the best tools like a Milwaukee Drill - and right now, thats copilot/gpt-4.
If you don't want to be tied to a company and like opensource, feel free to connect a toy motor to an AA battery to drill your holes... Or to use Llama/Stable Code 3B.
Re: Stable Code 3B: Coding on the Edge
#20I just tried this model with Koboldcpp on my LLM box. I got gibberish back. My prompt - "please show me how to write a web scraper in Python" The response? I've written my first ever python script about 5 months ago and I really don't remember anything except for the fact that I used Selenium in order to scrape websites (in this case, Google). So you can probably just copy/paste all of these lines from your own Pytho…
This doesn't seem like gibberish though?