Is anyone else here building AI programming services based on models like this? I see a lot of comments saying the models can't do much programming. But I just suspect there must be a silent contingent that is also working on services like that. And maybe less likely to promote the abilities of these models because it encourages competition.
SantaCoder: A new 1.1B code model for generation and infilling
11–20 of 76 posts
Re: SantaCoder: A new 1.1B code model for generation and infilling
#12Is anyone else here building AI programming services based on models like this? I see a lot of comments saying the models can't do much programming. But I just suspect there must be a silent contingent that is also working on services like that. And maybe less likely to promote the abilities of these models because it encourages competition.
We are at Codeium (codeium.com)! Not the SantaCoder model specifically, but the same types of LLM architectures. We've started with AI-based code autocomplete, but we think there is a lot more we can do.
We wrote up some of our learnings so far in @swyx's blog recently: https://lspace.swyx.io/p/what-building-copilot-for-x-really
Re: SantaCoder: A new 1.1B code model for generation and infilling
#13I am having trouble getting the demo to run. It just errors out
Give this notebook a shot:
https://github.com/arjunguha/BigCode-demos/blob/main/bigcode...
A GPU will help, but I found it passable on a CPU as well.
Re: SantaCoder: A new 1.1B code model for generation and infilling
#14I am having trouble getting the demo to run. It just errors out
The demo is up again!
Re: SantaCoder: A new 1.1B code model for generation and infilling
#15Earlier quoted context omitted.
Might be overloaded – if you have a GPU you can try running it locally by getting the model weights here: https://huggingface.co/bigcode/santacoder
Any idea how much GPU memory you'd need to run this locally? EDIT: just tried it and it didn't seem to go past ~6gb
It's 1 billion model with Fp16 precision so 4-6 GB max.
Re: SantaCoder: A new 1.1B code model for generation and infilling
#16Is anyone else here building AI programming services based on models like this? I see a lot of comments saying the models can't do much programming. But I just suspect there must be a silent contingent that is also working on services like that. And maybe less likely to promote the abilities of these models because it encourages competition.
We are at Codeium (codeium.com)! Not the SantaCoder model specifically, but the same types of LLM architectures. We've started with AI-based code autocomplete, but we think there is a lot more we can do. We wrote up some of our learnings so far in @swyx's blog recently: https://lspace.swyx.io/p/what-building-copilot-for-x-really
What I would really like is something I saw someone talking about here; I'd like the editor to brighten text it finds "unexpected" which could immediately alert to bugs, or to the fact that the code I'm writing looks weird in some way and might either be restructured or accompanied by a comment.
Re: SantaCoder: A new 1.1B code model for generation and infilling
#17Increase the number of tokens to a large number and you end up with masterpieces like this:
def all_elements_in_range_excluding_and_including_and_excluding_and_including_and_excluding(sequence, start, end):
Re: SantaCoder: A new 1.1B code model for generation and infilling
#18Re: SantaCoder: A new 1.1B code model for generation and infilling
#19Earlier quoted context omitted.
We are at Codeium (codeium.com)! Not the SantaCoder model specifically, but the same types of LLM architectures. We've started with AI-based code autocomplete, but we think there is a lot more we can do. We wrote up some of our learnings so far in @swyx's blog recently: https://lspace.swyx.io/p/what-building-copilot-for-x-really
What I would really like is something I saw someone talking about here; I'd like the editor to brighten text it finds "unexpected" which could immediately alert to bugs, or to the fact that the code I'm writing looks weird in some way and might either be restructured or accompanied by a comment.
Yep, these kinds of applications are on our mind! We consider autocomplete to be the "baseline" task since there are plenty of benchmarks and research to compare our model's performance to, but there's lots of things like highlighting code, upgrading to new libraries/conventions, etc that we can do with a good base model.
Re: SantaCoder: A new 1.1B code model for generation and infilling
#20Is anyone else here building AI programming services based on models like this? I see a lot of comments saying the models can't do much programming. But I just suspect there must be a silent contingent that is also working on services like that. And maybe less likely to promote the abilities of these models because it encourages competition.
We built a semantic code search CLI tool (fully local and open source) using a similar model that I tuned https://github.com/sturdy-dev/semantic-code-search