131–140 of 166 posts
Mistral really became what all the other over-hyped EU AI start-ups / collectives (Stability, Eleuther, Aleph Alpha, Nyonic, possibly Black Forest Labs, government-funded collaborations, ...) failed to achieve, although many of them existed way before Mistral. Congrats to them, great work.
Earlier quoted context omitted.
How do you determine peak memory usage? Just look at activity monitor? I've yet to find a good overview of how much memory each model needs for different context lengths (other than back of the envelope #weights * bits). LM Studio warns you if a model will likely not fit, but it's not very exact.
MLX reports peak memory usage at the end of the response. Otherwise I'll use Activity Monitor.
Still, it reports accurate peak memory usage for tensors living on GPU, but seems to miss some of the non-Metal overhead, however small (https://github.com/aukejw/mlx_transformers_benchmark/issues/...).
Earlier quoted context omitted.
Mistral isn't really Europe, it's France. Europe has some plans but as far as I can tell their goal isn't to make something that can really compete. The goal is to make EU data stay in the EU for businesses, meanwhile every user that is not forced by their company sends their data to the US or China.
Last I checked France is in Europe. It would be like saying Google or Apple are not American because they are in California.
The EU is more similar to NAFTA or five eyes, and culturally the loyalty is more similar to the US vs the anglosphere, like how Americans think of Australia, UK and Canada. Well, again, until recently. Things are changing fast.
While I am rooting for Mistral, having access to a diverse set of models is the killer app IMHO. Sometimes you want to code. Sometimes you want to write. Not all models are made equal.
I think all providers guarantee that they will not use your API inputs for training, it's meant as the pro version after all.
Plus it's dirt cheap, I query them several times per day, with access to high end thinking models, and pay just a few € per month.
Earlier quoted context omitted.
Couldn't you could place a very light weight model in front to figure out which model to use?
That’s a perfectly valid idea in theory, but in practice you’ll run into a few painful trade-offs, especially in multi-user environments. Trust me, I'm currently doing exactly that in our fairly limited exploration of how we can leverage local LLMs at work (SME). Unless you have sufficient VRAM to keep all potential specialized models loaded simultaneously (which negates some of the "lightweight" benefit for the over…
Too little too late, I work in a large European investment bank and we're already using Anthropic's Claude via Gitlab Duo.
Earlier quoted context omitted.
Simon, can you recommend some small models that would be usable for coding on a standard M4 Mac Mini (only 16G ram) ?
Strange idea, but if I'd like to set up a solid LLM for use in my home network, how much processing power would I need for a multi-purpose model? A Raspberry Pi? And old ThinkPad? A fully speced-out latest gen Macbook? edit: One of those old Mac Pros?
Mistral really became what all the other over-hyped EU AI start-ups / collectives (Stability, Eleuther, Aleph Alpha, Nyonic, possibly Black Forest Labs, government-funded collaborations, ...) failed to achieve, although many of them existed way before Mistral. Congrats to them, great work.
I’m wondering why. More funding, better talent, strategy, or something else?
While I am rooting for Mistral, having access to a diverse set of models is the killer app IMHO. Sometimes you want to code. Sometimes you want to write. Not all models are made equal.
Same here. Since I started using LLMs a bit more, the killer step for me was to set up API access to a variety of providers (Mistral, Anthropic, Gemini, OpenAI), and use a unified client to access them. I'm usually coding at the CLI, so I installed 'aichat' from github and it does an amazing job. Switch models on the fly, switch between one-shot and session mode, log everything locally for later access, and ask casua…
Earlier quoted context omitted.
I’m wondering why. More funding, better talent, strategy, or something else?
i'm an outsider but none of the startups mentioned above ever came to my ears. Mistral suddenly popped after openai/anthropic exploded, and they were rapidly described as the 3rd contender, with emphasis on technical merit. Maybe i was fooled though.