Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Codestral: Mistral's Code Model

mistral.ai

Post reply on HN
Print view View original

81–90 of 225 posts

Re: Codestral: Mistral's Code Model

#81

Post by JimDabell »

behnamoh wrote: post #72
analyte123 wrote: post #48

The license for this [1] prohibits use of the model and its outputs for any commercial activity, or even any "live" (whatever that means) conditions, commercial or not. There seems to be an exclusion for using the code outputs as part of "development". But wait! It also prohibits "any internal usage by employees in the context of the company's business activities". However you interpret these clauses, this puts their…

From the website: > licensed under the new Mistral AI Non-Production License, which means that you can use it for research and testing purposes. ... Which basically means "we give you this model. Go find its weaknesses and report on r/locallama. Then we'll use that to improve our commercial model which we won't open-source." I'm sick of abusing the word "open-source" in this field.

> I'm sick of abusing the word "open-source" in this field.

They don’t call this open source anywhere, do they? As far as I can see, they only say it’s open weights and that it’s available under their Mistral AI Non-Production License for research and testing. That doesn’t scream “open source” to me.

Re: Codestral: Mistral's Code Model

#82

Post by croes »

whiplash451 wrote: post #73
icedchai wrote: post #62

Earlier quoted context omitted.

I'm skeptical. I've run into people who used LLMs to code, then can't debug it without someone else's help. It may get you 80% there though.

It does not get you 80% there if it achieves what you described. It rather gets you 100% into trouble.

Programmer view vs management view.

100% of nothing vs 80% of enough.

That's the risk of AI. Not that AI outperforms humans already but that managers believe it does. That and that code writing is the main work of programmers.

Re: Codestral: Mistral's Code Model

#83

Post by nicce »

foobiekr wrote: post #65
analyte123 wrote: post #48

The license for this [1] prohibits use of the model and its outputs for any commercial activity, or even any "live" (whatever that means) conditions, commercial or not. There seems to be an exclusion for using the code outputs as part of "development". But wait! It also prohibits "any internal usage by employees in the context of the company's business activities". However you interpret these clauses, this puts their…

There's some irony in the fact that people will ignore this license in exactly the same way Mistral and all the other LLM guys ignore the copyright and licensing on the works they ingest.

In many countries you even can't claim copyright for the output of the AI to use license like this.

Re: Codestral: Mistral's Code Model

#84

Post by Closi »

TechDebtDevin wrote: post #76
Closi wrote: post #64

Earlier quoted context omitted.

Depends if benchmarking is the aim, rather than decreasing the time it takes to build things.

Well sure, but that wasn't what we were discussing. The original comment says they use that as their benchmark. While their coding task is a bit complex compared to other benchmarking prompts, it's not that crazy. Here is an example of prompts used for benchmarking with Python for reference: https://huggingface.co/datasets/mbpp?row=98 At the end of the day LLMs in their current iteration aren't intended to do even mo…

The original comment says nothing about benchmarking, they just say that an AI can’t one shot their complex task?

Re: Codestral: Mistral's Code Model

#85

Post by demosthanos »

JimDabell wrote: post #81
behnamoh wrote: post #72

Earlier quoted context omitted.

From the website: > licensed under the new Mistral AI Non-Production License, which means that you can use it for research and testing purposes. ... Which basically means "we give you this model. Go find its weaknesses and report on r/locallama. Then we'll use that to improve our commercial model which we won't open-source." I'm sick of abusing the word "open-source" in this field.

> I'm sick of abusing the word "open-source" in this field. They don’t call this open source anywhere, do they? As far as I can see, they only say it’s open weights and that it’s available under their Mistral AI Non-Production License for research and testing. That doesn’t scream “open source” to me.

They do say "open-weight", which is I think still very misleading in this context. Open-weight sounds like it should be the same as open-source, just for weights instead of the full source (for example, training data and the code used to generate the weights may not be released). This isn't really "open" in any meaningful sense.

Re: Codestral: Mistral's Code Model

#86

Post by gyudin »

JimDabell wrote: post #81
behnamoh wrote: post #72

Earlier quoted context omitted.

From the website: > licensed under the new Mistral AI Non-Production License, which means that you can use it for research and testing purposes. ... Which basically means "we give you this model. Go find its weaknesses and report on r/locallama. Then we'll use that to improve our commercial model which we won't open-source." I'm sick of abusing the word "open-source" in this field.

> I'm sick of abusing the word "open-source" in this field. They don’t call this open source anywhere, do they? As far as I can see, they only say it’s open weights and that it’s available under their Mistral AI Non-Production License for research and testing. That doesn’t scream “open source” to me.

All their other models are “open source” and it was the selling point they built their brand on. I doubt they made their new model completely different from previous ones so it’s supposed be open source too, unless they found some juridical loophole lol

Re: Codestral: Mistral's Code Model

#88

Post by das_keyboard »

rohansood15 wrote: post #70
analyte123 wrote: post #48

The license for this [1] prohibits use of the model and its outputs for any commercial activity, or even any "live" (whatever that means) conditions, commercial or not. There seems to be an exclusion for using the code outputs as part of "development". But wait! It also prohibits "any internal usage by employees in the context of the company's business activities". However you interpret these clauses, this puts their…

That license is just hilarious.

OT, but 7.2 reads like the description of some Yu-Gi-Oh card or something:

> Mistral AI may terminate this Agreement at any time [...]. Sections 5, 6, 7 and 8 shall survive the termination of this Agreement.

Re: Codestral: Mistral's Code Model

#89

Post by AnimalMuppet »

bongodongobob wrote: post #30
ddavis wrote: post #25

My favorite thing to ask the models designed for programming is: "Using Python write a pure ASGI middleware that intercepts the request body, response headers, and response body, stores that information in a dict, and then JSON encodes it to be sent to an external program using a function called transmit." None of them ever get it right :)

Cool, you've identified that your prompt is inadequate for the task. 'Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out?'

How is that "putting in wrong figures"? It's a perfectly valid prompt, written in clear, proper English.

Re: Codestral: Mistral's Code Model

#90

Post by gyudin »

ddavis wrote: post #25

My favorite thing to ask the models designed for programming is: "Using Python write a pure ASGI middleware that intercepts the request body, response headers, and response body, stores that information in a dict, and then JSON encodes it to be sent to an external program using a function called transmit." None of them ever get it right :)

I ask software developers to do the same thing and give them the same amount of time. None of them ever write a single line of code :)
Post reply on HN