Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Yi-Coder: A Small but Mighty LLM for Code

01-ai.github.io

Post reply on HN
Print view View original

81–90 of 117 posts

Re: Yi-Coder: A Small but Mighty LLM for Code

#81

Post by magnat »

smokel wrote: post #79

Does anyone know why the sizes of these models are typically expressed in number of weights (i.e 1.5B and 9B in this case), without mentioning the weight size in bytes? For practical reasons, I often like to know how much GPU RAM is required to run these models locally. The actual number of weights seems to only express some kind of relative power, which I doubt is relevant to most users. Edit: reformulated to sound…

Because you can quantize a model e.g. from original 16 bits down to 5 bits per weight to fit your available memory constraints.

Re: Yi-Coder: A Small but Mighty LLM for Code

#82

Post by tarruda »

JediPig wrote: post #59

I tested this out on my workload ( SRE/Devops/C#/Golang/C++ ). it started responding about non-sense on a simple write me boto python script that changes x ,y,z value. Then I tried other questions in my past to compare... However, I believe the engineer who did the LLM, just used the questions in benchmarks. One instance after a hour of use ( I stopped then ) it answered one question with 4 different programming lang…

Have you ran the model in full FP16? It is possible a lot of performance is lost when running quantized versions.

Re: Yi-Coder: A Small but Mighty LLM for Code

#83

Post by GaggiX »

smokel wrote: post #79

Does anyone know why the sizes of these models are typically expressed in number of weights (i.e 1.5B and 9B in this case), without mentioning the weight size in bytes? For practical reasons, I often like to know how much GPU RAM is required to run these models locally. The actual number of weights seems to only express some kind of relative power, which I doubt is relevant to most users. Edit: reformulated to sound…

The weight size depends on the accuracy you are running the model at, you usually do not run a model at fp16 as it would be wasteful.

Re: Yi-Coder: A Small but Mighty LLM for Code

#84

Post by tarruda »

gloosx wrote: post #75

Can someone explain these Aider benchmarks to me? They pass same 113 tests through llm every time. Why they then extrapolate ability of llm to pass these 113 basic python challenges to the general ability to produce/edit code? For me it sounds like this or that model is 70% accurate in solving same hundred python training tasks, but why does it mean that it's good at other languages and arbitrary, private tasks as we…

It seems this is the problem with most benchmarks, which is why benchmark performance doesn't mean much these days.

Re: Yi-Coder: A Small but Mighty LLM for Code

#86

Post by coolspot »

zeroq wrote: post #85

Everytime someone tells how AI 10x his programming capabilities I'm like "tell me you're bad at coding without telling me".

It allows me to move much faster, because I can write a comment describing something more high-level and get plausible code from it to review & correct.

Re: Yi-Coder: A Small but Mighty LLM for Code

#87

Post by Gracana »

tmikaeld wrote: post #63
JediPig wrote: post #59

I tested this out on my workload ( SRE/Devops/C#/Golang/C++ ). it started responding about non-sense on a simple write me boto python script that changes x ,y,z value. Then I tried other questions in my past to compare... However, I believe the engineer who did the LLM, just used the questions in benchmarks. One instance after a hour of use ( I stopped then ) it answered one question with 4 different programming lang…

I have the same experience, hallucinates and rambles on and on about "solutions" that are not related. Unfortunately, this has always been my experience with all open source code models that can be self-hosted.

It sounds like you are trying to chat with the base model when you should be using a chat model.

Re: Yi-Coder: A Small but Mighty LLM for Code

#88

Post by IshKebab »

viraptor wrote: post #45
rty32 wrote: post #38

Earlier quoted context omitted.

Coding Assistants are not good enough (yet). Inline suggestions and chats are incredibly helpful and boost productivity (and only to those who know to use them well), but that's as fast as they go today. If they can take a Jira ticket, debug the code, create a patch for a large codebase and understand and respect all the workarounds in a legacy codebase, I would have a problem with it.

Have you seen https://www.swebench.com/ ? Once you engage agentic behaviour, it can take you way further than just the chats. We're already in the "resolving JIRA tickets" area - it's just hard to setup, not very well known, and may be expensive.

> We're already in the "resolving JIRA tickets" area

For very simple tasks maybe, but not for the kinds of things I get paid to do.

I don't think it will be able to get to the level of reliably doing difficult programming tasks that require understanding and inferring requirements without having AGI, in which case society has other things to worry about than programmers losing their jobs.

Re: Yi-Coder: A Small but Mighty LLM for Code

#89

Post by smcleod »

bubblyworld wrote: post #17
smcleod wrote: post #2

Weird they're comparing it to really old deepseek v1 models, even v2 has been out a long time now.

My barely-informed guess is that they don't have the resources to run it (it's a 200b+ model).

That’s for the larger model, most people running it locally use the -lite model (both of which has lots of benchmarks published)

Re: Yi-Coder: A Small but Mighty LLM for Code

#90

Post by deely3 »

rfoo wrote: post #8
theshrike79 wrote: post #3

> Continue pretrained on 2.4 Trillion high-quality tokens over 52 major programming languages. I'm still waiting for a model that's highly specialised for a single language only - and either a lot smaller than these jack of all trades ones or VERY good at that specific language's nuances + libraries.

An unfortunate fact is, similar to human with infinite time, LLMs usually have better performance on your specific langauge when they are not limited to learn or over-sample one single language. Not unlike the common saying "learning to code in Haskell makes you a better C++ programmer". Of course, this is far from trivial, you don't just add more data and expect it to automatically be better for everything. So is ti…

> usually have better performance on your specific langauge when they are not limited to learn or over-sample one single language.

Source? Im very curious how learning one language helps model to generate code in language with different paradigms. Java, Markdown, JSON, HTML, Fortran?

Post reply on HN