Yi-Coder: A Small but Mighty LLM for Code • Hacker News Board

Re: Yi-Coder: A Small but Mighty LLM for Code

#91

Post by eropple » Thu, Sep 05, 2024, 9:06 PM UTC

rahimnathwani wrote: post #72

aithrowaway1987 wrote: post #69

Earlier quoted context omitted.

Look at who the tools are marketed towards. Writing software involves a lot of tedium, eye strain, and frustration, even for experts who have put in a lot of hours practicing, so LLMs are marketed to help developers make their jobs easier. This is not the case for art or music generators: they are marketed towards (and created by) laypeople with who want generic content and don't care about human artists. These syste…

What you say may be true about the simplest workflow: enter a prompt and get one or more finished images. But many people use diffusion models in a much more interactive way, doing much more of the editing by hand. The simplest case is to erase part of a generated image, and prompt to infill. But there are people who spend hours to get a single image where they want it.

This is true, and there's some really cool stuff there, but that's not who most of this is marketed at. Small wonder there's backlash from artists and people who appreciate artists when the stated value proposition is "render artists unemployed".

Re: Yi-Coder: A Small but Mighty LLM for Code

#92

Post by samstave » Thu, Sep 05, 2024, 9:14 PM UTC

bilekas wrote: post #73

yard2010 wrote: post #60

Earlier quoted context omitted.

Well, good luck prosecuting Winnie the Pooh

Their services will just be blocked in the EU instead.. we've seen it in Italy early on with ChatGPT..

When will we have token-flow-aware-networking gear... Surely NVIDIA and others are already doing special traffic shaping for tokenFlows?

Whats the current state of such tech/thought/standards/vendors?

Re: Yi-Coder: A Small but Mighty LLM for Code

#93

Post by patrick-fitz » Thu, Sep 05, 2024, 9:14 PM UTC

I'd be interested to see how it performs on https://www.swebench.com/

Using SWE-agent + Yi-Coder-9B-Chat.

Re: Yi-Coder: A Small but Mighty LLM for Code

#94

Post by suprjami » Thu, Sep 05, 2024, 9:47 PM UTC

samstave wrote: post #52

suprjami wrote: post #32

Earlier quoted context omitted.

If you have a project which supports OpenAI API keys, you can point it at a LocalAI instance: https://localai.io/ This is easy to get "working" but difficult to configure for specific tasks due to docs being lacking or contradictory.

Can you post screens/configs on how setup success? Or at least state what you configured toward and how?

The documentation gives a quick start and many examples of integration with OpenAI projects like a chatbot. That's all I did.

Re: Yi-Coder: A Small but Mighty LLM for Code

#95

Post by denizener » Thu, Sep 05, 2024, 11:52 PM UTC

Palmik wrote: post #31

The difference between (A) software engineers reacting to AI models and systems for programming and (B) artists (whether it's painters, musicians or otherwise) reacting to AI models for generating images, music, etc. is very interesting. I wonder what's the reason.

I love art and code, IMO is because Cursor is really good and AI art is not that good.

There isn't a good metaphor for the problem with AI art. I would say it is like some kind of chocolate cake that the first few bites seem like the best cake you have ever had and then progressive bites become more and more shit until you stop even considering eating it. Then at some point even the thought of the cake makes you want to puke.

I say this as someone who thought we reached the art singularity in December 2022. I have no philosophical or moral problem with AI art. It just kind of sucks.

Cursor/Sonnet on the other hand just blew my mind earlier today.

Re: Yi-Coder: A Small but Mighty LLM for Code

#96

Post by GaggiX » Fri, Sep 06, 2024, 12:04 AM UTC

denizener wrote: post #95

Palmik wrote: post #31

The difference between (A) software engineers reacting to AI models and systems for programming and (B) artists (whether it's painters, musicians or otherwise) reacting to AI models for generating images, music, etc. is very interesting. I wonder what's the reason.

I love art and code, IMO is because Cursor is really good and AI art is not that good. There isn't a good metaphor for the problem with AI art. I would say it is like some kind of chocolate cake that the first few bites seem like the best cake you have ever had and then progressive bites become more and more shit until you stop even considering eating it. Then at some point even the thought of the cake makes you want…

There are really good models for AI art, if people care. I think that AI is better at making an image from start to finish than making some software from start to finish.

And I use Claude 3.5 Sonnet myself.

Re: Yi-Coder: A Small but Mighty LLM for Code

#97

Post by samstave » Fri, Sep 06, 2024, 4:02 AM UTC

samstave wrote: post #92

bilekas wrote: post #73

Earlier quoted context omitted.

Their services will just be blocked in the EU instead.. we've seen it in Italy early on with ChatGPT..

When will we have token-flow-aware-networking gear... Surely NVIDIA and others are already doing special traffic shaping for tokenFlows? Whats the current state of such tech/thought/standards/vendors?

What about token sniffing? Is that even possible? Interesting? Valuable?

Re: Yi-Coder: A Small but Mighty LLM for Code

#98

Post by tmikaeld » Fri, Sep 06, 2024, 4:25 AM UTC

Gracana wrote: post #87

tmikaeld wrote: post #63

Earlier quoted context omitted.

I have the same experience, hallucinates and rambles on and on about "solutions" that are not related. Unfortunately, this has always been my experience with all open source code models that can be self-hosted.

It sounds like you are trying to chat with the base model when you should be using a chat model.

No, I’m using 9b-chat-q8_0 on a 4090

Re: Yi-Coder: A Small but Mighty LLM for Code

#99

Post by cztomsik » Fri, Sep 06, 2024, 6:43 AM UTC

deely3 wrote: post #90

rfoo wrote: post #8

Earlier quoted context omitted.

An unfortunate fact is, similar to human with infinite time, LLMs usually have better performance on your specific langauge when they are not limited to learn or over-sample one single language. Not unlike the common saying "learning to code in Haskell makes you a better C++ programmer". Of course, this is far from trivial, you don't just add more data and expect it to automatically be better for everything. So is ti…

> usually have better performance on your specific langauge when they are not limited to learn or over-sample one single language. Source? Im very curious how learning one language helps model to generate code in language with different paradigms. Java, Markdown, JSON, HTML, Fortran?

I think around the BLOOM models (2022) it was found out that if you train english-only, the model performs worse than if you have even little mixture of other languages.

Also, there were other papers (one epoch is all you need) where it was shown that diverse data is better than multiple epochs, and finally, there was paper (textbooks is all you need) for famous Phi model, with conclusion that high-quality data > lots of data.

This by itself is not a proof for your specific question but you can extrapolate.

Re: Yi-Coder: A Small but Mighty LLM for Code

#100

Post by tmikaeld » Fri, Sep 06, 2024, 6:59 AM UTC

tmikaeld wrote: post #98

Gracana wrote: post #87

Earlier quoted context omitted.

It sounds like you are trying to chat with the base model when you should be using a chat model.

No, I’m using 9b-chat-q8_0 on a 4090

Turns out that Ollama on windows will run multiple models in parallell consuming all available VRAM and RAM. Changing it to 1 fixed the issue, now it's working great! However, the context length for the output is very small - only 1024 tokens.