Yi-Coder: A Small but Mighty LLM for Code
01-ai.github.io
Yi-Coder: A Small but Mighty LLM for Code
1–10 of 117 posts
Re: Yi-Coder: A Small but Mighty LLM for Code
#2Weird they're comparing it to really old deepseek v1 models, even v2 has been out a long time now.
Re: Yi-Coder: A Small but Mighty LLM for Code
#3> Continue pretrained on 2.4 Trillion high-quality tokens over 52 major programming languages.
I'm still waiting for a model that's highly specialised for a single language only - and either a lot smaller than these jack of all trades ones or VERY good at that specific language's nuances + libraries.
Re: Yi-Coder: A Small but Mighty LLM for Code
#4Claude 3.5 Sonnet still holds the LLM crown for code which I'll use when wanting to check the output of the best LLM, however my Continue Dev, Aider and Claude Dev plugins are currently configured to use DeepSeek Coder V2 236B (and local ollama DeepSeek Coder V2 for tab completions) as it offers the best value at $0.14M/$0.28M which sits just below Claude 3.5 Sonnet on Aider's leaderboard [1] whilst being 43x cheaper.
Re: Yi-Coder: A Small but Mighty LLM for Code
#5Weird they're comparing it to really old deepseek v1 models, even v2 has been out a long time now.
My exact thoughts, especially because DeepseekV2 is meant to be a massive improvement.
It seems to be an emerging trend people should look out for that model release sheets often contain comparisons with out of date models and don't inform so much as just try to make the model look "best."
It's an annoying trend. Untrustworthy metrics betray untrustworthy morals.
Re: Yi-Coder: A Small but Mighty LLM for Code
#6> Continue pretrained on 2.4 Trillion high-quality tokens over 52 major programming languages. I'm still waiting for a model that's highly specialised for a single language only - and either a lot smaller than these jack of all trades ones or VERY good at that specific language's nuances + libraries.
If the LLM training makes the LLM generalize things between languages, then it is better to leave it like it is...
Re: Yi-Coder: A Small but Mighty LLM for Code
#7> Continue pretrained on 2.4 Trillion high-quality tokens over 52 major programming languages. I'm still waiting for a model that's highly specialised for a single language only - and either a lot smaller than these jack of all trades ones or VERY good at that specific language's nuances + libraries.
I’d be interested to know if that trade off ends up better. There’s probably a lot of useful training that transfers well between languages, so I wouldn’t be that surprised if the extra tokens helped across all languages. I would guess a top quality single language model would need to be very well supported, eg Python or JavaScript. Not, say, Clojure.
Re: Yi-Coder: A Small but Mighty LLM for Code
#8> Continue pretrained on 2.4 Trillion high-quality tokens over 52 major programming languages. I'm still waiting for a model that's highly specialised for a single language only - and either a lot smaller than these jack of all trades ones or VERY good at that specific language's nuances + libraries.
An unfortunate fact is, similar to human with infinite time, LLMs usually have better performance on your specific langauge when they are not limited to learn or over-sample one single language. Not unlike the common saying "learning to code in Haskell makes you a better C++ programmer".
Of course, this is far from trivial, you don't just add more data and expect it to automatically be better for everything. So is time management for us mere mortals.
Re: Yi-Coder: A Small but Mighty LLM for Code
#9Are coding LLMs trained with the help of interpreters?
Re: Yi-Coder: A Small but Mighty LLM for Code
#10> Continue pretrained on 2.4 Trillion high-quality tokens over 52 major programming languages. I'm still waiting for a model that's highly specialised for a single language only - and either a lot smaller than these jack of all trades ones or VERY good at that specific language's nuances + libraries.
Unclear how much of their coding knowledge is in the space of syntax/semantics of a given language and how much in the latent space that generalizes across languages and logic in general. If I were to guess I'd say 80% is in the latter for the larger capable models. Even very small models (like in Karpathy's famous RNN blog) will get syntax right but that is superficial knowledge.