Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Mistral ships Le Chat – enterprise AI assistant that can run on prem

mistral.ai

Post reply on HN
Print view View original

61–70 of 166 posts

Re: Mistral ships Le Chat – enterprise AI assistant that can run on prem

#61

Post by _bin_ »

fhd2 wrote: post #21
distances wrote: post #18

Earlier quoted context omitted.

I also kind of don't understand how it seems everyone is using AI for coding. I haven't had a client yet which would have approved any external AI usage. So I basically use them as search engines on steroids, but code can't go directly in or out.

You might be able to get your clients to sign something to allow usage, but if you don't, as you say, it doesn't seem wise to vibe code for them. For two reasons: 1. A typical contract transfers the rights to the work. The ownership of AI generated code is legally a wee bit disputed. If you modify and refactor generated code heavily it's probably fine, but if you just accept AI generated code en masse, making your cl…

This comes down to a question of what one can prove. NNs are necessary not explainable and none of this would have much evidence to show in court.

Re: Mistral ships Le Chat – enterprise AI assistant that can run on prem

#62

Post by drilbo »

omneity wrote: post #31
resource_waste wrote: post #14

Earlier quoted context omitted.

Expected this comment. Mistral has been consistently last place, or at least last place among ChatGPT, Claude, Llama, and Gemini/Gemma. I know this because I had to use a permissive license for a side project and I was tortured by how miserably bad Mistral was, and how much better every other LLM was. Need the best? ChatGPT Need local stuff? Llama(maybe Gemma) Need to do barely legal things that break most company's…

> Need local stuff? Llama(maybe Gemma) You probably want to replace Llama with Qwen in there. And Gemma is not even close. > Mistral has been consistently last place, or at least last place among ChatGPT, Claude, Llama, and Gemini/Gemma. Mistral held for a long time the position of "workhorse open-weights base model" and nothing precludes them from taking it again with some smart positioning. They might not currently…

>You probably want to replace Llama with Qwen in there. And Gemma is not even close.

Have you tried the latest, gemma3? I've been pretty impressed with it. Altho I do agree that qwen3 quickly overshadowed it, it seems too soon to dismiss it altogether. EG, the 3~4b and smaller versions of gemma seem to freak out way less frequently than similar param size qwen versions, tho I haven't been able to rule out quant and other factors in this just yet.

It's very difficult to fault anyone for not keeping up with the latest SOTA in this space. The fact we have several options that anyone can serviceably run, even on mobile, is just incredible.

Anyway, i agree that Mistral is worth keeping an eye on. They played a huge part in pushing the other players toward open weights and proving smaller models can have a place at the table. While I personally can't get that excited about a closed model, it's definitely nice to see they haven't tapped out.

Re: Mistral ships Le Chat – enterprise AI assistant that can run on prem

#63

Post by ATechGuy »

codingbot3000 wrote: post #3

I think this is a game changer, because data privacy is a legitimate concern for many enterprise users. Btw, you can also run Mistral locally within the Docker model runner on a Mac.

Why not use confidential computing based offerings like Azure's private inference for privacy concerns?

Re: Mistral ships Le Chat – enterprise AI assistant that can run on prem

#65

Post by jt_b »

nicce wrote: post #49
codingbot3000 wrote: post #3

I think this is a game changer, because data privacy is a legitimate concern for many enterprise users. Btw, you can also run Mistral locally within the Docker model runner on a Mac.

> Btw, you can also run Mistral locally within the Docker model runner on a Mac. Efficiently? I thought macOS does not have API so that Docker could use GPU.

I haven't/wouldn't use it because I have a decent K8S ollama/open-webui setup, but docker announced this a month ago: https://www.docker.com/blog/introducing-docker-model-runner

Re: Mistral ships Le Chat – enterprise AI assistant that can run on prem

#67

Post by ATechGuy »

kergonath wrote: post #7
codingbot3000 wrote: post #3

I think this is a game changer, because data privacy is a legitimate concern for many enterprise users. Btw, you can also run Mistral locally within the Docker model runner on a Mac.

> I think this is a game changer, because data privacy is a legitimate concern for many enterprise users. Indeed. At work, we are experimenting with this. Using a cloud platform is a non-starter for data confidentiality reasons. On-premise is the way to go. Also, they’re not American, which helps. > Btw, you can also run Mistral locally within the Docker model runner on a Mac. True, but you can do that only with thei…

Have you tried using private inference that uses GPU confidential computing from Nvidia?

Re: Mistral ships Le Chat – enterprise AI assistant that can run on prem

#68

Post by indigodaddy »

simonw wrote: post #58
codingbot3000 wrote: post #3

I think this is a game changer, because data privacy is a legitimate concern for many enterprise users. Btw, you can also run Mistral locally within the Docker model runner on a Mac.

There are plenty of other ways to run Mistral models on a Mac. I'm a big fan of Mistral Small 3.1. I've run that using both Ollama (easiest) and MLX. Here are the Ollama models: https://ollama.com/library/mistral-small3.1/tags - the 15GB one works fine. For MLX https://huggingface.co/mlx-community/Mistral-Small-3.1-24B-I... and https://huggingface.co/mlx-community/Mistral-Small-3.1-24B-I... should work, I use the 8bi…

Simon, can you recommend some small models that would be usable for coding on a standard M4 Mac Mini (only 16G ram) ?

Re: Mistral ships Le Chat – enterprise AI assistant that can run on prem

#69

Post by simonw »

indigodaddy wrote: post #68
simonw wrote: post #58

Earlier quoted context omitted.

There are plenty of other ways to run Mistral models on a Mac. I'm a big fan of Mistral Small 3.1. I've run that using both Ollama (easiest) and MLX. Here are the Ollama models: https://ollama.com/library/mistral-small3.1/tags - the 15GB one works fine. For MLX https://huggingface.co/mlx-community/Mistral-Small-3.1-24B-I... and https://huggingface.co/mlx-community/Mistral-Small-3.1-24B-I... should work, I use the 8bi…

Simon, can you recommend some small models that would be usable for coding on a standard M4 Mac Mini (only 16G ram) ?

That's pretty tough - the problem is that you need to have RAM left over to run actual applications!

Qwen 3 8B on MLX runs in just 5GB of RAM and can write basic code but I don't know if it would be good enough for anything interesting: https://simonwillison.net/2025/May/2/qwen3-8b/

Honestly though with that little memory I'd stick to running against hosted LLMs - Claude 3.7 Sonnet, Gemini 2.5 Pro, o4-mini are all cheap enough that it's hard to spend much money with them for most coding workflows.

Re: Mistral ships Le Chat – enterprise AI assistant that can run on prem

#70

Post by omneity »

drilbo wrote: post #62
omneity wrote: post #31

Earlier quoted context omitted.

> Need local stuff? Llama(maybe Gemma) You probably want to replace Llama with Qwen in there. And Gemma is not even close. > Mistral has been consistently last place, or at least last place among ChatGPT, Claude, Llama, and Gemini/Gemma. Mistral held for a long time the position of "workhorse open-weights base model" and nothing precludes them from taking it again with some smart positioning. They might not currently…

>You probably want to replace Llama with Qwen in there. And Gemma is not even close. Have you tried the latest, gemma3? I've been pretty impressed with it. Altho I do agree that qwen3 quickly overshadowed it, it seems too soon to dismiss it altogether. EG, the 3~4b and smaller versions of gemma seem to freak out way less frequently than similar param size qwen versions, tho I haven't been able to rule out quant and o…

It's probably subjective to your own use, but for me Gemma3 is not particularly usable (i.e. not competitive or delivering a particular value for me to make use of it).

Qwen 2.5 14B blows Gemma 27B out of the water for my use. Qwen 2.5 3B is also very competitive. The 3 series is even more interesting with the 0.6B model actually useful for basic tasks and not just a curiosity.

Where I find Qwen relatively lackluster is its complete lack of personality.

Post reply on HN