Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Mistral ships Le Chat – enterprise AI assistant that can run on prem

mistral.ai

Post reply on HN
Print view View original

121–130 of 166 posts

Re: Mistral ships Le Chat – enterprise AI assistant that can run on prem

#121

Post by tecleandor »

frabcus wrote: post #119
Havoc wrote: post #77

Not quite following. It seems to talk about features common associated with local servers but then ends with available on gcp Is this an API point? A model enterprises deploy locally? A piece of software plus a local model? There is so much corporate synergy speak there I can’t tell what they’re selling

They mention Google Cloud Marketplace (not Google Cloud Platform), this seems to be their listing there: https://console.cloud.google.com/marketplace/product/mistral... Which says: "Managed Services are fully hosted, managed and supported by the service providers. Although you register with the service provider to use the service, Google handles all billing." My assumption is that they're using Google Marketplace for…

Lota of tools offer billing you via Google Marketplace or the AWS equivalent as:

- it joins billing with other stuff

- I guess it's easier to get approval

- and more important (at least in our case), it allows you to reach your Google Cloud (or AWS) contract commitments of expense, and keep your discounts :)

Re: Mistral ships Le Chat – enterprise AI assistant that can run on prem

#122

Post by blitzar »

distances wrote: post #18
kergonath wrote: post #7

Earlier quoted context omitted.

> I think this is a game changer, because data privacy is a legitimate concern for many enterprise users. Indeed. At work, we are experimenting with this. Using a cloud platform is a non-starter for data confidentiality reasons. On-premise is the way to go. Also, they’re not American, which helps. > Btw, you can also run Mistral locally within the Docker model runner on a Mac. True, but you can do that only with thei…

I also kind of don't understand how it seems everyone is using AI for coding. I haven't had a client yet which would have approved any external AI usage. So I basically use them as search engines on steroids, but code can't go directly in or out.

I also kind of don't understand how it seems everyone is using AI for doing their homework. I haven't had a teacher yet which would have approved any AI usage.

Same process, less people being called out for "cheating" in a professional setting.

Re: Mistral ships Le Chat – enterprise AI assistant that can run on prem

#123

Post by tigroferoce »

atwrk wrote: post #107
_bin_ wrote: post #86

Earlier quoted context omitted.

It would be short-termist for Americans or euros to use chinese-made models. Increasing their popularity has an indirect but significant cost in the long term. china "winning AI" should be an unacceptable outcome for America or europe by any means necessary.

Why would that be? I can see why Americans wouldn't want to do that, but Europeans? In the current political climate, where the US openly claims their desire to annex European territory and so on? I'd rather see them prefer a locally hostable open source solution like DeepSeek.

My two cents, as European, is that since we are more and more asking to LLMs for information, it wouldn't be wise to let a foreign country, not even truly democratic, to choose the information we get.

Re: Mistral ships Le Chat – enterprise AI assistant that can run on prem

#124

Post by beernet »

Mistral really became what all the other over-hyped EU AI start-ups / collectives (Stability, Eleuther, Aleph Alpha, Nyonic, possibly Black Forest Labs, government-funded collaborations, ...) failed to achieve, although many of them existed way before Mistral. Congrats to them, great work.

Re: Mistral ships Le Chat – enterprise AI assistant that can run on prem

#125

Post by Tepix »

kergonath wrote: post #7
codingbot3000 wrote: post #3

I think this is a game changer, because data privacy is a legitimate concern for many enterprise users. Btw, you can also run Mistral locally within the Docker model runner on a Mac.

> I think this is a game changer, because data privacy is a legitimate concern for many enterprise users. Indeed. At work, we are experimenting with this. Using a cloud platform is a non-starter for data confidentiality reasons. On-premise is the way to go. Also, they’re not American, which helps. > Btw, you can also run Mistral locally within the Docker model runner on a Mac. True, but you can do that only with thei…

premises, not premise.

https://www.grammar-monster.com/easily_confused/premise_prem...

Re: Mistral ships Le Chat – enterprise AI assistant that can run on prem

#126

Post by promiseofbeans »

sschueller wrote: post #95
the_clarence wrote: post #90

Earlier quoted context omitted.

Tbh I think the one general model approach is winning. People don't want to figure out which model is better at what unless its for a very specific task.

Couldn't you could place a very light weight model in front to figure out which model to use?

I mean, the general purpose models already do this in a way, routing to a selected expert. It's a pretty fundamental concept for ensemble learning, which is what MOE experts are, effectively.

I don't see any reason you couldn't stack more layers of routing in front, to select the model. However, this starts to seem inefficient.

I think the optimal solution will eventually be companies training and publishing hyper-focused expert models, that are designed to be used with other models and a router. Then interface vendors can purchase different experts and assemble the models themselves, like how a phone manufacter purchases parts from many suppliers, even their compeditors, in order to create the best final product. The bigger players (e.g. Apple for this analogy) might make more parts in house, but even the latest iPhone still has Samsung chips in it in teardowns.

Re: Mistral ships Le Chat – enterprise AI assistant that can run on prem

#127

Post by the_other_mac »

indigodaddy wrote: post #68
simonw wrote: post #58

Earlier quoted context omitted.

There are plenty of other ways to run Mistral models on a Mac. I'm a big fan of Mistral Small 3.1. I've run that using both Ollama (easiest) and MLX. Here are the Ollama models: https://ollama.com/library/mistral-small3.1/tags - the 15GB one works fine. For MLX https://huggingface.co/mlx-community/Mistral-Small-3.1-24B-I... and https://huggingface.co/mlx-community/Mistral-Small-3.1-24B-I... should work, I use the 8bi…

Simon, can you recommend some small models that would be usable for coding on a standard M4 Mac Mini (only 16G ram) ?

Run Mistral 7b in under 4gb ram:

https://github.com/garagesteve1155/Overload

(As announced this morning in the FB group "Dull Men's Club!)

Re: Mistral ships Le Chat – enterprise AI assistant that can run on prem

#128

Post by croes »

_bin_ wrote: post #60
croes wrote: post #13

Earlier quoted context omitted.

Is there are replacement for the Safe Harbor replacement? Otherwise it could be illegal to transfer EU data to US companies

The law means don’t do what a slow moving regulator can and will prove in court. In this case, the law has no moral valence so I doubt anyone there would feel guilty breaking it. He may mean individuals are using ChatGPT unofficially even if prohibited nominally by management. Such is the case almost everywhere.

There is a difference if you upload your data or your customers data.

There are countries in the EU where you get sued for less

Re: Mistral ships Le Chat – enterprise AI assistant that can run on prem

#129

Post by simonw »

aukejw wrote: post #114
simonw wrote: post #89

Earlier quoted context omitted.

I really like Mistral Small 3.1 (I have a 64GB M2 as well). Qwen 3 is worth trying in different sizes too. I don't know if they'll be good enough for general coding tasks though - I've been spoiled by API access to Claude 3.7 Sonnet and o4-mini and Gemini 2.5 Pro.

How do you determine peak memory usage? Just look at activity monitor? I've yet to find a good overview of how much memory each model needs for different context lengths (other than back of the envelope #weights * bits). LM Studio warns you if a model will likely not fit, but it's not very exact.

MLX reports peak memory usage at the end of the response. Otherwise I'll use Activity Monitor.

Re: Mistral ships Le Chat – enterprise AI assistant that can run on prem

#130

Post by Lalabadie »

chedabob wrote: post #111
jychang wrote: post #85

Earlier quoted context omitted.

16GB on a mac with unified memory is too small for good coding models. Anything on that machine is severely compromised. Maybe in ~1 year we will see better models that fit in ~8gb vram, but not yet. Right now, for a coding LLM on a Mac, the standard is Qwen 3 32b, which runs great on any M1 mac with 32gb memory or better. Qwen 3 235b is better, but fewer people have 128gb memory. Anything smaller than 32b, you start…

What do you use to interface with Qwen? I have LMStudio installed, and use Continue in VSCode, but it doesn't feel nearly as feature rich compared to using something like Cursor's IDE, or the GitHub Copilot plugin.

Continue can be your autocomplete provider – and use a smaller and faster model. Something like Cline (or Roo or Kilocode or another fork) would be the more Cursor-like assistant there.
Post reply on HN