171–180 of 225 posts
Earlier quoted context omitted.
There's some irony in the fact that people will ignore this license in exactly the same way Mistral and all the other LLM guys ignore the copyright and licensing on the works they ingest.
In many countries you even can't claim copyright for the output of the AI to use license like this.
Earlier quoted context omitted.
I'm skeptical. I've run into people who used LLMs to code, then can't debug it without someone else's help. It may get you 80% there though.
I've run into working programmers who were bad at debugging before LLMs existed.
i've been noticing that there's a divergence in philosophy between Llama style LLMs (Mistral are Meta alums so I'm counting them in tehre) and OpenAI/GPT style LLMs when it comes to code. GPT3.5+ prioritized code very heavily - there's no CodeGPT, its just GPT4, and every version is better than the last. Whereas the Llama/Mistral models are now shipping the general language model first, then adding CodeLlama/Codestra…
Codex[1] is OpenAI's CodeGPT. It's what powers GitHub Copilot and it is very good but not publicly accessible. Maybe they don't want something else to outcompete Copilot.
Earlier quoted context omitted.
> Who cares? It seems you're not allowed to integrate this with anything else and show it to anyone, even as an art project. Now they just lack the means to enforce it.
impossible to enforce
How effective these techniques are and how acceptable as a proof it is is yet to be defined.
I don't think it is the case here, they probably don't really care, and watermarking has a cost.
Earlier quoted context omitted.
I would argue the opposite is true. My experience with coding with LLMs is that the only thing it's really good at is generating boilerplate that it has more-or-less seen before (essentially a library, even if is somewhat adapted), however it is incapable of the creative thinking that developers regularly need to engage in when architecting a solution for their use case.
My experience is the opposite. When I started using Copilot I thought it would only be good at standard boilerplate but I'm constantly surprised how well it understands my completely convoluted legacy architecture that barely I understand myself even though I'm the only contributor.
The license for this [1] prohibits use of the model and its outputs for any commercial activity, or even any "live" (whatever that means) conditions, commercial or not. There seems to be an exclusion for using the code outputs as part of "development". But wait! It also prohibits "any internal usage by employees in the context of the company's business activities". However you interpret these clauses, this puts their…
From the website: > licensed under the new Mistral AI Non-Production License, which means that you can use it for research and testing purposes. ... Which basically means "we give you this model. Go find its weaknesses and report on r/locallama. Then we'll use that to improve our commercial model which we won't open-source." I'm sick of abusing the word "open-source" in this field.
This is maybe a debatable claim, but I’ll contend that without the magnificent rebel who leaked the original LLaMA weights the last, what, 15 months would have gone completely differently.
The legislators and courts and lawyers will be years if not decades sorting all this out.
For now there seems to be a productive if slightly uneasy truce: outside of a few groups at a few firms, everyone seems to be maximizing for innovation and generally behaving under a positive sum expectation.
One imagines if some really cool tune of this model shows up as a magnet or even on huggingface, the courteous thing probably happened: Mistral was notified in advance and some mutually beneficial arrangement was agreed to in outline, maybe inked, maybe not.
I don’t work for Mistral, so that’s pure speculation, but the big company I spent most of my career at would have certainly said “can we hire this person? can we buy this company? can we collaborate with people who do awesome stuff with our stuff that we didn’t think of?”
The icky actors kind of dominate the headlines and I’m as guilty as anyone and guiltier than most of letting that be top of mind too often.
In the large this is really cool and kind of new.
I’m personally rather optimistic that we’re well past the point when outright piracy or flagrantly adversarial license violations are either necessary or useful.
To me this license seems like an invitation to build on Mistral’s work and approach them with the results, and given how well a posture of openness with some safeguards is working out for FAIR and the LLaMA group, that’s certainly the outcome I’d be hoping for in their position.
Maybe open AI was an unrealistic goal. Maybe AvailableAI is what we wind up with, and that wouldn’t be too bad.
How does this compare to Github Copilot? It's not shown in their comparison
This is an open weights 22B model. The download on Huggingface is 44GB. Is there a rule-of-thumb estimate for how much RAM this would need to be used locally? Is the RAM requirement the same for a GPU and "unified" RAM like Apple silicon?
Running LLM models on a MacBook Pro with Apple Silicon vs. a PC with an Nvidia 4090 GPU has trade-offs. My 128GB MacBook Pro handles models using up to 96GB of unified memory, running at a little under half the speed of a 4090. If you use a quantized version of full floating point model, you can run the largest open models available.
While the 4090 has 24GB of dedicated memory and higher bandwidth (1000 GB/s vs. 400 GB/s on M3 Max), the Mac’s unified memory system (up to 128GB) is flexible and holds smarter models (8 bit and 6 bit models act still mostly all there, 4 bit is so so, 2 bit is brain damaged).
The M2 Ultra in Mac Studio offers even more (800 GB/s bandwidth and 192GB memory). So, ok, 6 or 8 of 4090 cards or 4 x A6000 cards excels in raw performance, but Apple’s unified memory in a laptop fits in your backback.
It's not clear to me why Macbooks and Mac Studio Ultras with maxed out RAM aren't selling better if you look at the convenience and price relative to model size. Models that fit in one 4090 or even a pair of 4090s are toys compared to what fits on these, so for the big models you're comparing a laptop to a minifridge.
How does this compare to Github Copilot? It's not shown in their comparison
It is fast all right, but the quality is not there. I asked it to implement OAuth with Stytch and Ktor and it made everything up. I pointed out the correct name for the package and asked if it really knew the SDK, and it apologized and repeated the same made up code after merely changing the name of the package.
We have a big list of example questions we get from devs trying us out and we've tested several home grown and third party providers and thus far haven't seen anything good enough that we'd put into production.
Thanks for testing this out for us! I'll cross it off our list :)
Earlier quoted context omitted.
Break your prompt up into smaller pieces and it can.
Taken to the extreme, a sufficiently broken down prompt is simply the code itself. The whole point is to prompt less?