The MBPP column should bold DeepSeek as it has a better score than Codestral.
codegeex4-all-9b beats them "on paper" so that's why it's not in the benchmarks.
Codestral Mamba
101–110 of 148 posts
Re: Codestral Mamba
#102Earlier quoted context omitted.
Great info. Do you also know the state of the code assistants? Any thoughts on copilot versus others?
I've been following the state of things, but I'm not sure which ones are the best. There's Meta's CodeLlama[1], Mistral's Codestral[2], DeepSeek AI's DeepSeek-Coder-V2-Instruct[3], CodeGemma[4], Alibaba's CodeQwen[5], and Microsoft's WizardCoder[6]. I'm pretty sure CodeLlama is out of date now. I've heard DeepSeek LLMs are good and DeepSeek-Coder-V2-Instruct was released recently. With the good reputation and its mas…
try THUDM/codegeex4-all-9b
Re: Codestral Mamba
#103Earlier quoted context omitted.
On top of Hacker News (the target demographic for coders) without an effective monetizable call to action? What a missed opportunity. Github Copilot makes +100M/year, if not way way more. Having a VS Code extension for Mistral would be a revenue stream if it was one-click and better or cheaper than Github Copilot. It is malpractice in my mind to not be doing this if you are investing in creating coding models.
How the hell does Copilot make $100M/yr? That seems an order of magnitude higher than I would expect at the high end.
I was thinking the opposite...remember there's enterprise subscriptions and multi-million dollar contracts with single companies.
Re: Codestral Mamba
#104weird they compare to deepseek-coder v1.5 when we already have v2.0. Any advantage to use codestral mamba apart from that it's lighter in weights?
deepseek-coder 2.0 is big, even lite version.
I do wish they compare it to codegeex4-all-9b
Re: Codestral Mamba
#105I know it's just a throwaway line, but the bit about Cleopatra at the top feels in poor taste. It's completely inaccurate in that no one has ever attributed her death to a "mamba", and even the asp that some sources claim has been disputed. But even aside from that, it just feels weird that a human being's death has turned into a random reference you can make in a throwaway joke while advertising a product. They're c…
Too soon!
Rick and Morty ref?
Re: Codestral Mamba
#106Earlier quoted context omitted.
On top of Hacker News (the target demographic for coders) without an effective monetizable call to action? What a missed opportunity. Github Copilot makes +100M/year, if not way way more. Having a VS Code extension for Mistral would be a revenue stream if it was one-click and better or cheaper than Github Copilot. It is malpractice in my mind to not be doing this if you are investing in creating coding models.
How the hell does Copilot make $100M/yr? That seems an order of magnitude higher than I would expect at the high end.
???
Re: Codestral Mamba
#107Is this the active Codestral model on Le Chat? I got quite some mixed results from it tonight.
Re: Codestral Mamba
#108> Unlike Transformer models, Mamba models offer the advantage of linear time inference and the theoretical ability to model sequences of infinite length
> We have tested Codestral Mamba on in-context retrieval capabilities up to 256k tokens
Why only 256k tokens? Gemini's context window is 1 million or more and it's (probably) not even using Mamba.
Re: Codestral Mamba
#109Does anyone have a video or written article that would get one up to speed with a bit of the history/progression and current products that are out there for one to try locally? This is coming from someone that understands the general concepts of how LLMs work but only used the general publicly available tools like ChatGPT, Claude, etc. I want to see if I have any hardware I can stress and run something locally, but d…
Here's a summary of what's happened the past couple of years and what tools are out there. After ChatGPT released, there was a lot of hype in the space but open source was far behind. Iirc the best open foundation LLM that existed was GPT-2 but it was two generations behind. Awhile later Meta released LLaMA[1], a well trained base foundation model, which brought an explosion to open source. It was soon implemented in…
Overall a good write up, but I have a few quips:
> Awhile later Meta released LLaMA[1],
I think Stable Diffusion was first to release a SOTA model (August 2022) that worked locally, not in language but image generation, but it set the tone for Meta. LLaMA only came in February 2023.
> The company Mistral had proven itself in the past with very impressive LLaMA finetunes
Mistal is not a finetune of LLaMA, it is a model trained from scratch. Also, Mistral was most of the time better than LLaMA during this period.
> Quantization techniques improved which meant LLaMA was able to run with less and less RAM with greater and greater accuracy
Quantization does not improve accuracy, except if you trade off precision for longer context maybe, but not on similar prompts. It is like JPEG compression, the original is always better for a specific image, but for the same byte size you get more resolution from JPEG than say... a PNG.
Re: Codestral Mamba
#110Earlier quoted context omitted.
Most the 7b instruct models are very bad outside very simple queries. You can run a 7b on most modern hardware.How fast will vary. To run 30-70b models you're getting in the realm of needing 24gb or more of vRAM.
>Most the 7b instruct models are very bad outside very simple queries. I can't agree with "very bad". Maybe your standards are set by the best, largest models, but have a little perspective: a modern 7b model is a friggin magical piece of software. Fully in the realm of sci-fi until basically last Tuesday. It can reliably summarize documents, bash a 30 minute rambling voice note into a terse proposal, and give you so…
Even better, install ollama and then do "ollama run llama3", it works like docker, pulls the model locally and starts a chat session right there in the terminal. No need to compile. Or just run the docker image "ollama/ollama".