Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Mistral NeMo

mistral.ai

Post reply on HN
Print view View original

31–40 of 165 posts

Re: Mistral NeMo

#31

Post by miki123211 »

jorgesborges wrote: post #18

I’m AI stupid. Does anyone know if training on multiple languages provides “cross-over” — so training done in German can be utilized when answering a prompt in English? I once went through various Wikipedia articles in a couple languages and the differences were interesting. For some reason I thought they’d be almost verbatim (forgetting that’s not how Wikipedia works!) and while I can’t remember exactly I felt they…

Generally yes, with caveats.

There was some research showing that training a model on facts like "the mother of John Smith is Alice" but in German allowed it to answer questions like "who's the mother of John Smith", but not questions like "what's the name of Alice's child", regardless of language. Not sure if this holds at larger model sizes though, it's the sort of problem that's usually fixable by throwing more parameters at it.

Language models definitely do generalize to some extend and they're not "stochastic parrots" as previously thought, but there are some weird ways in which we expect them to generalize but they don't.

Re: Mistral NeMo

#32

Post by knicholes »

MikeKusold wrote: post #27
madeofpalk wrote: post #20

I find it interesting how coding/software development still appears to be the one category that these most popular models release specialised models for. Where's the finance or legal models from Mistral or Meta or OpenAI? Perhaps it's just confirmation bias, but programming really does seem to be the ideal usecase for LLMs in a way that other professions just haven't been able to crack. Compared to other types of wor…

Those are regulated industries, where as software development is not. An AI spitting back bad code won't compile. An AI spitting back bad financial/legal advice bankrupts people.

Generally I agree! I saw a guy shamefully admit he didn't read the output carefully enough when using generated code (that ran), but there was a min() instead of a max(), and it messed up a month of his metrics!

Re: Mistral NeMo

#33

Post by Raed667 »

pixelatedindex wrote: post #14

Pardon me if this is a dumb question, but is it possible for me to download these models into my computer (I have a 1080ti and a [2|3]070ti) and generate some sort of api interface? That way I can write programs that calls this API, and I find this appealing. EDIT: This a 1W light bulb moment for me, thank you!

First thing I did when i saw the headline was to look for it on ollma but it didn't land there yet: https://ollama.com/library?sort=newest&q=NeMo

Re: Mistral NeMo

#34

Post by mcemilg »

I believe that if Mistral is serious about advancing in open source, they should consider sharing the corpus used for training their models, at least the base models pretraining data.

Re: Mistral NeMo

#35

Post by a2128 »

simonw wrote: post #26

I wonder why Mistral et al don't prepare GGUF versions of these for launch day? If I were them I'd want to be the default source of the versions of my models that people use, rather than farming that out to whichever third party races to publish the GGUF (and other formats) first.

llama.cpp is still under development and they sometimes come out with breaking changes or new quantization methods, and it can be a lot of work to keep up with these changes as you publish more models over time. It's easier to just publish a standard float32 safetensors that works with PyTorch, and let the community deal with other runtimes and file formats.

If it's a new architecture, then there's also additional work needed to add support in llama.cpp, which means more dev time, more testing, and potentially loss of surprise model release if the development work has to be done out in the open

Re: Mistral NeMo

#36

Post by exe34 »

yjftsjthsd-h wrote: post #3

> Today, we are excited to release Mistral NeMo, a 12B model built in collaboration with NVIDIA. Mistral NeMo offers a large context window of up to 128k tokens. Its reasoning, world knowledge, and coding accuracy are state-of-the-art in its size category. As it relies on standard architecture, Mistral NeMo is easy to use and a drop-in replacement in any system using Mistral 7B. > We have released pre-trained base an…

tensors look about 20gb. not sure what that's like in vram.

Re: Mistral NeMo

#37

Post by Raed667 »

mythz wrote: post #10
pants2 wrote: post #6

Exciting, I think 12B is the sweet spot for running locally - large enough to be useful, fast enough to run on a decent laptop.

IMO Google's Gemma2 27B [1] is the sweet spot for running locally on commodity 16GB VRAM cards. [1] https://ollama.com/library/gemma2:27b

If I "only" have 16GB of ram on a macbook pro, would that still work ?

Re: Mistral NeMo

#38

Post by sakesun »

madeofpalk wrote: post #20

I find it interesting how coding/software development still appears to be the one category that these most popular models release specialised models for. Where's the finance or legal models from Mistral or Meta or OpenAI? Perhaps it's just confirmation bias, but programming really does seem to be the ideal usecase for LLMs in a way that other professions just haven't been able to crack. Compared to other types of wor…

Generating code has significant economical benefit. The code once generated can be execute so many times without requiring high computing resources, unlike AI model.

Re: Mistral NeMo

#39

Post by dannyw »

voiper1 wrote: post #11
Workaccount2 wrote: post #5

Is "Parameter Creep" going to becomes a thing? They hold up Llama-8b as a competitor despite NeMo having 50% more parameters. The same thing happened with gemma-27b, where they compared it to all the 7-9b models. It seems like an easy way to boost benchmarks while coming off as "small" at first glance.

Oddly, they are only charging slightly more for their hosted version: open-mistral-7b is 25c/m tokens open-mistral-nemo-2407 is 30c/m tokens https://mistral.ai/technology/#pricing

Possibly a NVIDIA subsidy. You run NEMO models, you get cheaper GPUs.

Re: Mistral NeMo

#40

Post by alecco »

saberience wrote: post #17

Two questions: 1) Anyone have any idea of VRAM requirements? 2) When will this be available on ollama?

The model has 12 billion parameters and uses FP8, so 1 byte each. With some working memory I'd bet you can run it on 24GB.

> Designed to fit on the memory of a single NVIDIA L40S, NVIDIA GeForce RTX 4090 or NVIDIA RTX 4500 GPU

Post reply on HN