Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Codestral: Mistral's Code Model

mistral.ai

Post reply on HN
Print view View original

111–120 of 225 posts

Re: Codestral: Mistral's Code Model

#111

Post by dieortin »

gyudin wrote: post #90
ddavis wrote: post #25

My favorite thing to ask the models designed for programming is: "Using Python write a pure ASGI middleware that intercepts the request body, response headers, and response body, stores that information in a dict, and then JSON encodes it to be sent to an external program using a function called transmit." None of them ever get it right :)

I ask software developers to do the same thing and give them the same amount of time. None of them ever write a single line of code :)

Give an LLM all the time you want, and they will still not get it right. In fact, they most likely will give worse and worse answers with time. That’s a big difference with a software developer.

Re: Codestral: Mistral's Code Model

#112

Post by localfirst »

meiraleal wrote: post #92
analyte123 wrote: post #48

The license for this [1] prohibits use of the model and its outputs for any commercial activity, or even any "live" (whatever that means) conditions, commercial or not. There seems to be an exclusion for using the code outputs as part of "development". But wait! It also prohibits "any internal usage by employees in the context of the company's business activities". However you interpret these clauses, this puts their…

> Who cares? It seems you're not allowed to integrate this with anything else and show it to anyone, even as an art project. Now they just lack the means to enforce it.

impossible to enforce

Re: Codestral: Mistral's Code Model

#114

Post by hannasanarion »

nicce wrote: post #83
foobiekr wrote: post #65

Earlier quoted context omitted.

There's some irony in the fact that people will ignore this license in exactly the same way Mistral and all the other LLM guys ignore the copyright and licensing on the works they ingest.

In many countries you even can't claim copyright for the output of the AI to use license like this.

Copyright on the software that produces something isn't the same as copyright on the output.

The library's copyright is intact, as normal, and they can control who uses it and how just like any other software.

The output of AI systems is not copyrightable, but the systems themselves are, and associated EULAs are valid.

Re: Codestral: Mistral's Code Model

#115

Post by belter »

foobiekr wrote: post #65
analyte123 wrote: post #48

The license for this [1] prohibits use of the model and its outputs for any commercial activity, or even any "live" (whatever that means) conditions, commercial or not. There seems to be an exclusion for using the code outputs as part of "development". But wait! It also prohibits "any internal usage by employees in the context of the company's business activities". However you interpret these clauses, this puts their…

There's some irony in the fact that people will ignore this license in exactly the same way Mistral and all the other LLM guys ignore the copyright and licensing on the works they ingest.

And nobody will sue anybody because suing means...discovery....

Re: Codestral: Mistral's Code Model

#116

Post by achierius »

meiraleal wrote: post #97
qeternity wrote: post #93

Earlier quoted context omitted.

Taken to the extreme, a sufficiently broken down prompt is simply the code itself. The whole point is to prompt less?

> Taken to the extreme, a sufficiently broken down prompt is simply the code itself it is not. But the artifacts generated through the steps will be code. The last prompt will have most of the code supplied to it as the context.

A prompt is just a specification for an output. Code is just what we call a sufficiently detailed specification.

Re: Codestral: Mistral's Code Model

#117

Post by TeMPOraL »

Zambyte wrote: post #101
demosthanos wrote: post #85

Earlier quoted context omitted.

They do say "open-weight", which is I think still very misleading in this context. Open-weight sounds like it should be the same as open-source, just for weights instead of the full source (for example, training data and the code used to generate the weights may not be released). This isn't really "open" in any meaningful sense.

The fact that I can downloaded it and run it myself is a pretty meaningful amount of openness to me. I can easily ignore their bogus claims about what I'm allowed to do with it due to their distribution model. I can't necessarily do the same with a propriety service, as they can cut me off if the way I use the output makes them sad :(

> The fact that I can downloaded it and run it myself is a pretty meaningful amount of openness to me

That's typically called freeware, though.

Re: Codestral: Mistral's Code Model

#118

Post by whoami_nr »

memothon wrote: post #18
swyx wrote: post #13

i've been noticing that there's a divergence in philosophy between Llama style LLMs (Mistral are Meta alums so I'm counting them in tehre) and OpenAI/GPT style LLMs when it comes to code. GPT3.5+ prioritized code very heavily - there's no CodeGPT, its just GPT4, and every version is better than the last. Whereas the Llama/Mistral models are now shipping the general language model first, then adding CodeLlama/Codestra…

>Zuck has mentioned recently That's a really surprising thing to hear, where did you see that? The only quote I've seen is this one: >“One hypothesis was that coding isn’t that important because it’s not like a lot of people are going to ask coding questions in WhatsApp,” he says. “It turns out that coding is actually really important structurally for having the LLMs be able to understand the rigor and hierarchical s…

He mentioned it on the Dwarkesh podcast: https://www.youtube.com/watch?v=bc6uFV9CJGg

Re: Codestral: Mistral's Code Model

#119

Post by [deleted] »

analyte123 wrote: post #48

The license for this [1] prohibits use of the model and its outputs for any commercial activity, or even any "live" (whatever that means) conditions, commercial or not. There seems to be an exclusion for using the code outputs as part of "development". But wait! It also prohibits "any internal usage by employees in the context of the company's business activities". However you interpret these clauses, this puts their…

[deleted]

Re: Codestral: Mistral's Code Model

#120

Post by afro88 »

jasonjmcghee wrote: post #38
sashank_1509 wrote: post #12

Seems nice but some preliminary testing against GPT-4o shows it’s lacking a bit. It does a pretty good job for easy questions though

GPT-4o is really oddly hit or miss for code. Sometimes it outperforms GPT-4 in quality by a fair amount, and other times it starts repeating itself. Duplicating function definitions, even misremembering what things are named. It seems to have to do with length. If the output exceeds a few thousand tokens, it seems to experience some pretty bad failure modes.

4o can only output 4k tokens. So the training to complete an answer within 4k tokens is probably kicking in and nerfing the quality
Post reply on HN