Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Mistral ships Le Chat – enterprise AI assistant that can run on prem

mistral.ai

Post reply on HN
Print view View original

111–120 of 166 posts

Re: Mistral ships Le Chat – enterprise AI assistant that can run on prem

#111

Post by chedabob »

jychang wrote: post #85
indigodaddy wrote: post #68

Earlier quoted context omitted.

Simon, can you recommend some small models that would be usable for coding on a standard M4 Mac Mini (only 16G ram) ?

16GB on a mac with unified memory is too small for good coding models. Anything on that machine is severely compromised. Maybe in ~1 year we will see better models that fit in ~8gb vram, but not yet. Right now, for a coding LLM on a Mac, the standard is Qwen 3 32b, which runs great on any M1 mac with 32gb memory or better. Qwen 3 235b is better, but fewer people have 128gb memory. Anything smaller than 32b, you start…

What do you use to interface with Qwen?

I have LMStudio installed, and use Continue in VSCode, but it doesn't feel nearly as feature rich compared to using something like Cursor's IDE, or the GitHub Copilot plugin.

Re: Mistral ships Le Chat – enterprise AI assistant that can run on prem

#112

Post by dzhiurgis »

codingbot3000 wrote: post #3

I think this is a game changer, because data privacy is a legitimate concern for many enterprise users. Btw, you can also run Mistral locally within the Docker model runner on a Mac.

How many is many? Literally all of them use cloud services.

Re: Mistral ships Le Chat – enterprise AI assistant that can run on prem

#113

Post by jagermo »

curiousgal wrote: post #8

Too little too late, I work in a large European investment bank and we're already using Anthropic's Claude via Gitlab Duo.

AI data residency is an issue for several of our customers, so I think there is still a big enough market for this.

Re: Mistral ships Le Chat – enterprise AI assistant that can run on prem

#114

Post by aukejw »

simonw wrote: post #89
codetrotter wrote: post #84

Earlier quoted context omitted.

How about on an MacBook Pro M2 Max with 64GB RAM? Any recommendations for local models for coding on that? I tried to run some of the differently sized DeepSeek R1 locally when those had recently come out, but couldn’t manage at the time to run any of them. And I had to download a lot of data to try those. So if you know a specific size of DeepSeek R1 that will work on 64GB RAM on MacBook Pro M2 Max, or another great…

I really like Mistral Small 3.1 (I have a 64GB M2 as well). Qwen 3 is worth trying in different sizes too. I don't know if they'll be good enough for general coding tasks though - I've been spoiled by API access to Claude 3.7 Sonnet and o4-mini and Gemini 2.5 Pro.

How do you determine peak memory usage? Just look at activity monitor?

I've yet to find a good overview of how much memory each model needs for different context lengths (other than back of the envelope #weights * bits). LM Studio warns you if a model will likely not fit, but it's not very exact.

Re: Mistral ships Le Chat – enterprise AI assistant that can run on prem

#115

Post by _pdp_ »

the_clarence wrote: post #90
_pdp_ wrote: post #10

While I am rooting for Mistral, having access to a diverse set of models is the killer app IMHO. Sometimes you want to code. Sometimes you want to write. Not all models are made equal.

Tbh I think the one general model approach is winning. People don't want to figure out which model is better at what unless its for a very specific task.

IMHO people want to interact with agents that do things not with models that chat. And agents by definition are specialised which means a specific model and Mistral might not be good for all types of tasks just like the top of line models are not always for everything.

Re: Mistral ships Le Chat – enterprise AI assistant that can run on prem

#116

Post by disgruntledphd2 »

victorbjorklund wrote: post #75
_mlbt wrote: post #6

Earlier quoted context omitted.

> our world-class AI engineering team offers support all the way through to value delivery.

Guess that makes sense. But I'm sure they charge good money for it and then you could just use that money for someone helping you with an open source model.

Presumably one throat to choke logic applies here, particularly in Europe.

Re: Mistral ships Le Chat – enterprise AI assistant that can run on prem

#117

Post by _joel »

mistrial9 wrote: post #81
jstummbillig wrote: post #59

Earlier quoted context omitted.

Nobody is seriously disputing the ownership of AI generated code. A serious dispute would be a considerable, concerted effort to stop AI code generation in any jurisdiction, that provides a contrast to the enormous , ongoing efforts by multiple large players with eye-watering investments to make code generation bigger and better. Note, that this is not a statement about the fairness or morality of LLM building, but t…

this is "Kool-aid" from the supply side of LLMs for coding IMO. Plenty of people are plenty upset about the capture of code at Github corral, fed into BigCorp$ training systems. parent statement reminds me of smug French in a castle north of London circa 1200, with furious locals standing outside the gates, dressed in rags with farm tools as weapons. One well-equipped tower guard says to another "no one is seriously…

Your mother was a hamster and your father smelt of elderberries?

Re: Mistral ships Le Chat – enterprise AI assistant that can run on prem

#118

Post by rowanajmarshall »

tootie wrote: post #82
starik36 wrote: post #66

I don't see any mention of hardware requirements for on prem. What GPUs? How many? Disk space?

I'm guessing it's flexible. Mistral makes small models capable of running on consumer hardware so they can probably scale up and down based on needs. And what is available from hosts.

I run a Mistral model on my phone!

Re: Mistral ships Le Chat – enterprise AI assistant that can run on prem

#119

Post by frabcus »

Havoc wrote: post #77

Not quite following. It seems to talk about features common associated with local servers but then ends with available on gcp Is this an API point? A model enterprises deploy locally? A piece of software plus a local model? There is so much corporate synergy speak there I can’t tell what they’re selling

They mention Google Cloud Marketplace (not Google Cloud Platform), this seems to be their listing there:

https://console.cloud.google.com/marketplace/product/mistral...

Which says:

"Managed Services are fully hosted, managed and supported by the service providers. Although you register with the service provider to use the service, Google handles all billing."

My assumption is that they're using Google Marketplace for discovery and billing, and they offer a hosted option or an on-prem option.

But agreed, it isn't clear!

Re: Mistral ships Le Chat – enterprise AI assistant that can run on prem

#120

Post by blitzar »

fortifAI wrote: post #103
guerrilla wrote: post #9

Interesting. Europe is really putting up a fight for once. I'm into it.

Mistral isn't really Europe, it's France. Europe has some plans but as far as I can tell their goal isn't to make something that can really compete. The goal is to make EU data stay in the EU for businesses, meanwhile every user that is not forced by their company sends their data to the US or China.

Last I checked France is in Europe. It would be like saying Google or Apple are not American because they are in California.
Post reply on HN