Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Codestral Mamba

mistral.ai

Post reply on HN
Print view View original

141–148 of 148 posts

Re: Codestral Mamba

#141

Post by theptip »

razodactyl wrote: post #133
solarkraft wrote: post #112

I kinda just want something that can keep up with the original version of Copilot. It was so much better than the crap they’re pumping out now (keeps messing up syntax and only completing a few characters at a time).

Supposedly they were training on feedback provided by the plugin itself but that approach doesn't make sense to me because: - I don't remember the shortcuts most of the time. - When I run completions I double take and realise they're wrong. - I am not a good source of data. All this information is being fed back into the model as positive feedback. So perhaps reason for it to have gone downhill. I recall it being ama…

It could also be that back in the day they were training with a bit more code than they should have been (eg private repos) and now the lawyers are more involved the training set is smaller/more sanitized.

Pure speculation of course.

Re: Codestral Mamba

#142

Post by ygouzerh »

nabakin wrote: post #76
culopatin wrote: post #3

Does anyone have a video or written article that would get one up to speed with a bit of the history/progression and current products that are out there for one to try locally? This is coming from someone that understands the general concepts of how LLMs work but only used the general publicly available tools like ChatGPT, Claude, etc. I want to see if I have any hardware I can stress and run something locally, but d…

Here's a summary of what's happened the past couple of years and what tools are out there. After ChatGPT released, there was a lot of hype in the space but open source was far behind. Iirc the best open foundation LLM that existed was GPT-2 but it was two generations behind. Awhile later Meta released LLaMA[1], a well trained base foundation model, which brought an explosion to open source. It was soon implemented in…

Wow very useful comment, thank you very much for all the work to write it!

Re: Codestral Mamba

#144

Post by thot_experiment »

trissi wrote: post #139
thot_experiment wrote: post #66

Does anyone have a favorite FIM capable model? I've been using codellama-13b through ollama w/ a vim extension i wrote and it's okay but not amazing, I definitely get better code most of the time out of Gemma-27b but no FIM (and for some reason codellama-34b has broken inference for me)

I use deepseek-coder-7b-instruct-v1.5 & DeepSeek-Coder-V2-Lite-Instruct when I want speed & codestral-22B-v0.1 when I want smartness. All of those are FIM capable, but especially deepseek-v2-lite is very picky with its prompt template so make sure you use it correctly... Depending on your hardware codestral-22B might be fast enough for everything, but for me it's a bit to slow... If you can run it deepseek v2 non-lig…

IIRC the codestral fim tokens aren't properly implemented in llama.cpp/ollama, what backend are you using to run them? id probably have to drop down to iq2_xxs or something for the full fat deepseek but I'll definitely look into codestral, I'm a big fan of mixtral, hopefully a MoE code model with FIM comes along soon.

EDIT: nvm, my mistake looks like it works fine https://github.com/ollama/ollama/issues/5403

Re: Codestral Mamba

#145

Post by antifa »

bhouston wrote: post #8

What are the steps required to get this running in VS Code? If they had linked to the instructions in their post (or better yet a link to a one click install of a VS Code Extension), it would help a lot with adoption. (BTW I consider it malpractice that they are at the top of hacker news with a model that is of great interest to a large portion of the users where and they do not have a monetizable call to action on t…

Maybe not this model, but checkout TabbyML for offline/selfhostws LLMs in vscode.

Re: Codestral Mamba

#146

Post by sfsylvester »

antifa wrote: post #145
bhouston wrote: post #8

What are the steps required to get this running in VS Code? If they had linked to the instructions in their post (or better yet a link to a one click install of a VS Code Extension), it would help a lot with adoption. (BTW I consider it malpractice that they are at the top of hacker news with a model that is of great interest to a large portion of the users where and they do not have a monetizable call to action on t…

Maybe not this model, but checkout TabbyML for offline/selfhostws LLMs in vscode.

Also looks like an older version of Codestral works well with TabbyML: https://tabby.tabbyml.com/blog/2024/07/09/tabby-codestral/

Thank you for sharing, this is almost exactly what I've been looking for, for ages!

Re: Codestral Mamba

#147

Post by rjurney »

mountainriver wrote: post #56
rjurney wrote: post #50

Earlier quoted context omitted.

Claude is much better. Overwhelmingly better. It not only implements deep learning models for me, it has great suggestions on evolving them to actually work.

lol no it’s not, the benchmarks don’t show that at all. Both have issues in different ways

Which benchmarks are you looking at? It is very competitive with GPT4o in the table of metrics I just built at work. Have you used it to code? Qualitatively, it is much better - once it can execute Python it will be supzors.
Post reply on HN