Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Codestral: Mistral's Code Model

mistral.ai

Post reply on HN
Print view View original

41–50 of 225 posts

Re: Codestral: Mistral's Code Model

#41

Post by guyomes »

swyx wrote: post #13

i've been noticing that there's a divergence in philosophy between Llama style LLMs (Mistral are Meta alums so I'm counting them in tehre) and OpenAI/GPT style LLMs when it comes to code. GPT3.5+ prioritized code very heavily - there's no CodeGPT, its just GPT4, and every version is better than the last. Whereas the Llama/Mistral models are now shipping the general language model first, then adding CodeLlama/Codestra…

> OpenAI is betting heavily on code as a way to improve LLM reasoning for AGI.

And researchers from Google Deepmind, University of Wisconsin-Madison and Laboratoire de l’Informatique du Parallélisme, University of Lyon, actually publish some of their results in that direction [1,2].

[1]: https://deepmind.google/discover/blog/funsearch-making-new-d...

[2]: https://www.nature.com/articles/s41586-023-06924-6

Re: Codestral: Mistral's Code Model

#42

Post by Rastonbury »

swyx wrote: post #13

i've been noticing that there's a divergence in philosophy between Llama style LLMs (Mistral are Meta alums so I'm counting them in tehre) and OpenAI/GPT style LLMs when it comes to code. GPT3.5+ prioritized code very heavily - there's no CodeGPT, its just GPT4, and every version is better than the last. Whereas the Llama/Mistral models are now shipping the general language model first, then adding CodeLlama/Codestra…

I thought that was the idea, open source small specific models that most people can run vs general purpose ones that require a massive amount of GPUs

Re: Codestral: Mistral's Code Model

#43

Post by meiraleal »

ddavis wrote: post #25

My favorite thing to ask the models designed for programming is: "Using Python write a pure ASGI middleware that intercepts the request body, response headers, and response body, stores that information in a dict, and then JSON encodes it to be sent to an external program using a function called transmit." None of them ever get it right :)

Interesting. My favorite thing to ask the models is to refactor code I've not touched for too long and this works very well.

Re: Codestral: Mistral's Code Model

#44

Post by jasonjmcghee »

rohansood15 wrote: post #16
mousetree wrote: post #2

How does this compare to Github Copilot? It's not shown in their comparison

Copilot primarily uses GPT-3.5, which is outclassed by Llama3-70B. And this model claims to be slightly better than Llama3-70B. Edit: For those who don't believe me, https://github.com/microsoft/vscode-copilot-release/issues/6... . Gpt-4 for chat, 3.5 for code.

GitHub Copilot uses GPT-3.5?

I was under the impression it was a custom codex model with a surrogate local model as per https://github.blog/2023-02-14-github-copilot-now-has-a-bett...

When did this change?

Re: Codestral: Mistral's Code Model

#45

Post by bongodongobob »

TechDebtDevin wrote: post #34
bongodongobob wrote: post #30

Earlier quoted context omitted.

Cool, you've identified that your prompt is inadequate for the task. 'Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out?'

Damn, show us your brilliant prompt then. LLMs cannot do this, not even in python, of which there are libraries like Blacksheep that honestly make it a trivial task.

My point is that you shouldn't expect to one shot everything. Have it start by writing a spec, then outline classes and methods, then write the code, and feed it debug stuff.

Re: Codestral: Mistral's Code Model

#46

Post by TechDebtDevin »

andruby wrote: post #10

This is an open weights 22B model. The download on Huggingface is 44GB. Is there a rule-of-thumb estimate for how much RAM this would need to be used locally? Is the RAM requirement the same for a GPU and "unified" RAM like Apple silicon?

I'm honestly not sure on how to measure the amount of vRAM required for these models but I suspect this would run relatively fast, depending on your use case, on a mid to high end 20 or 30 series card. No idea about Apple unified RAM. I get a lot out of performance out of even older cards such as a 1080ti but haven't tested this model.

Re: Codestral: Mistral's Code Model

#47

Post by croes »

IMTDb wrote: post #28

Is there a way to use this within VSCode like copilot , meaning having the "shadow code" appear while you code instead of having to tho back-and-forth between the editor and a chat-like interface ? For me, a significant component of the quality of these tools resides on the "client" side; being able to engineer a prompt that will yield to accurate code being generated by the model. The prompt needs to find and embed…

You mean like in their example VS code integration shown here?:

https://m.youtube.com/watch?v=mjltGOJMJZA

Re: Codestral: Mistral's Code Model

#48

Post by analyte123 »

The license for this [1] prohibits use of the model and its outputs for any commercial activity, or even any "live" (whatever that means) conditions, commercial or not.

There seems to be an exclusion for using the code outputs as part of "development". But wait! It also prohibits "any internal usage by employees in the context of the company's business activities". However you interpret these clauses, this puts their claims and comparisons on completely unequal ground. They only compare to other open-weight models, not GPT-4 or Opus, but a normal company or individual can do whatever they want with the Llama weights and outputs. LangChain? "Your favourite coding and building environment"? Who cares? It seems you're not allowed to integrate this with anything else and show it to anyone, even as an art project.

[1] https://mistral.ai/licenses/MNPL-0.1.md

Re: Codestral: Mistral's Code Model

#49

Post by jacekm »

IMTDb wrote: post #28

Is there a way to use this within VSCode like copilot , meaning having the "shadow code" appear while you code instead of having to tho back-and-forth between the editor and a chat-like interface ? For me, a significant component of the quality of these tools resides on the "client" side; being able to engineer a prompt that will yield to accurate code being generated by the model. The prompt needs to find and embed…

The article says that the model is available in Tabnine, a direct competitor to Copilot.

Re: Codestral: Mistral's Code Model

#50

Post by TechDebtDevin »

sharbloop wrote: post #21
fnbr wrote: post #15

Earlier quoted context omitted.

The rule of thumb is roughly 44gb, as most models are trained in bf16, and require 16 bits per parameter, so 2 bytes. You need a bit more for activations, so maybe 50GB? you need enough RAM and HBM (GPU RAM) so it’s a constraint on both.

Which GPU card can I buy to run this model? Can it run on commercial RTX3090 or does it need a custom GPU?

Easy..
Post reply on HN