Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Mistral ships Le Chat – enterprise AI assistant that can run on prem

mistral.ai

Post reply on HN
Print view View original

71–80 of 166 posts

Re: Mistral ships Le Chat – enterprise AI assistant that can run on prem

#71

Post by reichardt »

indigodaddy wrote: post #68
simonw wrote: post #58

Earlier quoted context omitted.

There are plenty of other ways to run Mistral models on a Mac. I'm a big fan of Mistral Small 3.1. I've run that using both Ollama (easiest) and MLX. Here are the Ollama models: https://ollama.com/library/mistral-small3.1/tags - the 15GB one works fine. For MLX https://huggingface.co/mlx-community/Mistral-Small-3.1-24B-I... and https://huggingface.co/mlx-community/Mistral-Small-3.1-24B-I... should work, I use the 8bi…

Simon, can you recommend some small models that would be usable for coding on a standard M4 Mac Mini (only 16G ram) ?

With around 4.6 GiB model size the new Qwen3-8B quantized to 4-bit should fit comfortably in 16 GiB of memory: https://huggingface.co/mlx-community/Qwen3-8B-4bit

Re: Mistral ships Le Chat – enterprise AI assistant that can run on prem

#73

Post by nicce »

jt_b wrote: post #65
nicce wrote: post #49

Earlier quoted context omitted.

> Btw, you can also run Mistral locally within the Docker model runner on a Mac. Efficiently? I thought macOS does not have API so that Docker could use GPU.

I haven't/wouldn't use it because I have a decent K8S ollama/open-webui setup, but docker announced this a month ago: https://www.docker.com/blog/introducing-docker-model-runner

Hmm, I guess that is not actually running inside container/ there is no isolation. Some kind of new way that mixes llama.cpp , OCI format and docker CLI.

Re: Mistral ships Le Chat – enterprise AI assistant that can run on prem

#74

Post by reverius42 »

jstummbillig wrote: post #59
fhd2 wrote: post #21

Earlier quoted context omitted.

You might be able to get your clients to sign something to allow usage, but if you don't, as you say, it doesn't seem wise to vibe code for them. For two reasons: 1. A typical contract transfers the rights to the work. The ownership of AI generated code is legally a wee bit disputed. If you modify and refactor generated code heavily it's probably fine, but if you just accept AI generated code en masse, making your cl…

Nobody is seriously disputing the ownership of AI generated code. A serious dispute would be a considerable, concerted effort to stop AI code generation in any jurisdiction, that provides a contrast to the enormous , ongoing efforts by multiple large players with eye-watering investments to make code generation bigger and better. Note, that this is not a statement about the fairness or morality of LLM building, but t…

> Nobody is seriously disputing the ownership of AI generated code

From what I've been following it seems very likely that, at least in the US, AI-generated anything can't actually be copyrighted and thus can't have ownership at all! The legal implications of this are yet to percolate through the system though.

Re: Mistral ships Le Chat – enterprise AI assistant that can run on prem

#75

Post by victorbjorklund »

_mlbt wrote: post #6
victorbjorklund wrote: post #4

Why use this instead of an open source model?

> our world-class AI engineering team offers support all the way through to value delivery.

Guess that makes sense. But I'm sure they charge good money for it and then you could just use that money for someone helping you with an open source model.

Re: Mistral ships Le Chat – enterprise AI assistant that can run on prem

#76

Post by lolinder »

codingbot3000 wrote: post #3

I think this is a game changer, because data privacy is a legitimate concern for many enterprise users. Btw, you can also run Mistral locally within the Docker model runner on a Mac.

Game changer feels a bit strong. This is a new entry in a field that's already pretty crowded with open source tooling that's already available to anyone with the time and desire to wire it all up. It's likely that they execute this better than the community-run projects have so far and make it more approachable and Enterprise friendly, but just for reference I have most of the features that they've listed here already set up on my desktop at home with Ollama, Open WebUI, and a collection of small hand-rolled apps that plug into them. I can't run very big models on mine, obviously, but if I were an Enterprise I would.

The key thing they'd need to nail to make this better than what's already out there is the integrations. If they can make it seamless to integrate with all the key third-party enterprise systems then they'll have something strong here, otherwise it's not obvious how much they're adding over Open WebUI, LibreChat, and the other self-hosted AI agent tooling that's already available.

Re: Mistral ships Le Chat – enterprise AI assistant that can run on prem

#77

Post by Havoc »

Not quite following. It seems to talk about features common associated with local servers but then ends with available on gcp

Is this an API point? A model enterprises deploy locally? A piece of software plus a local model?

There is so much corporate synergy speak there I can’t tell what they’re selling

Re: Mistral ships Le Chat – enterprise AI assistant that can run on prem

#78

Post by staunton »

reverius42 wrote: post #74
jstummbillig wrote: post #59

Earlier quoted context omitted.

Nobody is seriously disputing the ownership of AI generated code. A serious dispute would be a considerable, concerted effort to stop AI code generation in any jurisdiction, that provides a contrast to the enormous , ongoing efforts by multiple large players with eye-watering investments to make code generation bigger and better. Note, that this is not a statement about the fairness or morality of LLM building, but t…

> Nobody is seriously disputing the ownership of AI generated code From what I've been following it seems very likely that, at least in the US, AI-generated anything can't actually be copyrighted and thus can't have ownership at all! The legal implications of this are yet to percolate through the system though.

Only if that interpretation lasts despite likely intense lobbying to the contrary.

Re: Mistral ships Le Chat – enterprise AI assistant that can run on prem

#79

Post by abujazar »

codingbot3000 wrote: post #3

I think this is a game changer, because data privacy is a legitimate concern for many enterprise users. Btw, you can also run Mistral locally within the Docker model runner on a Mac.

Actually you shouldn't be running LLMs in Docker on Mac because it doesn't have GPU support. So the larger models will be extremely slow if they'll even produce a single token.

Re: Mistral ships Le Chat – enterprise AI assistant that can run on prem

#80

Post by abujazar »

distances wrote: post #18
kergonath wrote: post #7

Earlier quoted context omitted.

> I think this is a game changer, because data privacy is a legitimate concern for many enterprise users. Indeed. At work, we are experimenting with this. Using a cloud platform is a non-starter for data confidentiality reasons. On-premise is the way to go. Also, they’re not American, which helps. > Btw, you can also run Mistral locally within the Docker model runner on a Mac. True, but you can do that only with thei…

I also kind of don't understand how it seems everyone is using AI for coding. I haven't had a client yet which would have approved any external AI usage. So I basically use them as search engines on steroids, but code can't go directly in or out.

You can set up your IDE to use local LLMs through e.g. Ollama if your computer is powerful enough to run a decent model.
Post reply on HN