Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Codestral: Mistral's Code Model

mistral.ai

Post reply on HN
Print view View original

141–150 of 225 posts

Re: Codestral: Mistral's Code Model

#141

Post by alephxyz »

JimDabell wrote: post #75
ddavis wrote: post #25

My favorite thing to ask the models designed for programming is: "Using Python write a pure ASGI middleware that intercepts the request body, response headers, and response body, stores that information in a dict, and then JSON encodes it to be sent to an external program using a function called transmit." None of them ever get it right :)

I normally ask about building a multi-tenant system using async SQLAlchemy 2 ORM where some tables are shared between tenants in a global PostgreSQL schema and some are in a per-tenant schema. Nothing gets it right first time, but when ChatGPT 4 first came out, I could talk to it more and it would eventually get it right. Not long after that though, ChatGPT degraded. It would get it wrong on the first try, but with e…

>It would get it wrong on the first try, but with every subsequent follow up it would forget one of the constraints. Then when it was prompted to fix that one, it forgot a different one. And eventually it would cycle through all of the constraints, getting at least one wrong each time.

That drives me nuts and makes me ragequit about half the time. Although it's usually more effective to go and correct your initial prompt rather than prompt it again

Re: Codestral: Mistral's Code Model

#142

Post by piskov »

isaacrolandson wrote: post #135

Will this run on an M3 48GB?

You’ll need 44GB just for the weights

By default only 75% of unified memory is available to GPU if you have >36GB. So with 48 total only 36 is available for GPU with is lower than 44.

tldr; without quantization you will not be able to run it.

Re: Codestral: Mistral's Code Model

#143

Post by mijoharas »

pyepye wrote: post #37
IMTDb wrote: post #28

Is there a way to use this within VSCode like copilot , meaning having the "shadow code" appear while you code instead of having to tho back-and-forth between the editor and a chat-like interface ? For me, a significant component of the quality of these tools resides on the "client" side; being able to engineer a prompt that will yield to accurate code being generated by the model. The prompt needs to find and embed…

Not using Codestral (yet) but check out Continue.dev[1] with Ollama[2] running llama3:latest and starcoder2:3b. It gives you a locally running chat and edit via llama3 and autocomplete via starcoder2. It's not perfect but it's getting better and better. [1] https://www.continue.dev/ [2] https://ollama.com/

Wow... That site (continue.dev) managed to consistently crash my mobile google chrome.

I've had the odd crash now and again, but I can't think of many sites that will reliably make it hard crash. It's almost impressive.

Re: Codestral: Mistral's Code Model

#144

Post by hehdhdjehehegwv »

htrp wrote: post #139
hehdhdjehehegwv wrote: post #131

Earlier quoted context omitted.

So basically I, as an open source author, had my code eaten up by Mistral without my consent, but if I want to use their code model I’m subject to a bunch of restrictions that benefit their bottom line? The problem these AI companies have is they live in a glass house and they can’t throw IP rocks around without breaking their own “your content is our training data” foundation. They only reason I can think of that Go…

call it an Enterprise poison pill.

But a pill they also have to swallow.

Re: Codestral: Mistral's Code Model

#145

Post by boulos »

demosthanos wrote: post #85
JimDabell wrote: post #81

Earlier quoted context omitted.

> I'm sick of abusing the word "open-source" in this field. They don’t call this open source anywhere, do they? As far as I can see, they only say it’s open weights and that it’s available under their Mistral AI Non-Production License for research and testing. That doesn’t scream “open source” to me.

They do say "open-weight", which is I think still very misleading in this context. Open-weight sounds like it should be the same as open-source, just for weights instead of the full source (for example, training data and the code used to generate the weights may not be released). This isn't really "open" in any meaningful sense.

This is why I prefer the term "weights available" just like "source available". It makes it clear that you can get your hands on the copy, you could run this exact thing locally if they go out of business, etc. but it is definitely not open in the OSS sense.

Re: Codestral: Mistral's Code Model

#146

Post by spmurrayzzz »

ddavis wrote: post #25

My favorite thing to ask the models designed for programming is: "Using Python write a pure ASGI middleware that intercepts the request body, response headers, and response body, stores that information in a dict, and then JSON encodes it to be sent to an external program using a function called transmit." None of them ever get it right :)

I love to ask it to "make me a Node.js library that pings an ipv4 address, but you must use ZERO dependencies, you must only the native Node.js API modules"

The majority of models (both proprietary and open-weight) don't understand:

- by inference, ping means we're talking about ICMP

- ICMP requires raw sockets

- Node.js has no native raw socket API

You can do some CoT trickery to help it reason about the problem and maybe finally get it settled on a variety of solutions (usually some flavor of building a native add-on using C/C++/Rust/Go), or just guide it there step by step yourself, but the back and forth to get there requires a ton of pre-knowledge of the problem space which sorta defeats the purpose. If you just feed it the errors you get verbatim trying to run the code it generates, you end up in painful feedback loops.

(Note: I never expect the models to get this right, it's just a good microcosmic but concrete example of where knowledge & reasoning meets actual programming acumen, so its cool to see how models evolve to get better, if at all, at the task).

Re: Codestral: Mistral's Code Model

#147

Post by jdorfman »

elmariachi wrote: post #129
esafak wrote: post #91

Are there any IDE plugins that index your entire code base in order to provide contextual responses AND let you pick between the latest models? If not, consider it a product idea ;)

Cody by Sourcegraph allows you to do this. It doesn't have Codestral yet but probably will soon.

We are working on it.

Re: Codestral: Mistral's Code Model

#148

Post by IshKebab »

hehdhdjehehegwv wrote: post #131
foobiekr wrote: post #65

Earlier quoted context omitted.

There's some irony in the fact that people will ignore this license in exactly the same way Mistral and all the other LLM guys ignore the copyright and licensing on the works they ingest.

So basically I, as an open source author, had my code eaten up by Mistral without my consent, but if I want to use their code model I’m subject to a bunch of restrictions that benefit their bottom line? The problem these AI companies have is they live in a glass house and they can’t throw IP rocks around without breaking their own “your content is our training data” foundation. They only reason I can think of that Go…

> So basically I, as an open source author, had my code eaten up by Mistral without my consent

Not necessarily. You consented to people reading your code and learning from it when you posted it on Github. Whether or not there's an issue with AI doing the same remains to be settled. It certainly isn't clear cut that separate consent would be required.

Post reply on HN