Did anyone try to check how are it's multilingual skills vs. Gemma 2? On the page, it's compared with LLama 3 only.
Mistral NeMo
101–110 of 165 posts
Re: Mistral NeMo
#102Re: Mistral NeMo
#103What's the reason for measuring the model size in context window length and not GB? Also, are these small models OSS? Easier self hosting seems to be the main benefo for small models.
> Also, are these small models OSS?
From the very first paragraph on the page:
> released under the Apache 2.0 license.
Re: Mistral NeMo
#104Re: Mistral NeMo
#105Pardon me if this is a dumb question, but is it possible for me to download these models into my computer (I have a 1080ti and a [2|3]070ti) and generate some sort of api interface? That way I can write programs that calls this API, and I find this appealing. EDIT: This a 1W light bulb moment for me, thank you!
llama.cpp supports multi gpu across local network
https://www.reddit.com/r/LocalLLaMA/comments/1cyzi9e/llamacp...
and expose an OpenAI compatible server, or you can use their python bindings
Re: Mistral NeMo
#106Earlier quoted context omitted.
if you want to be lazy, 7b = 7gb of vRAM, 12b = 12gb of vRAM, but quantizing you might be able to do with with ~6-8. So any 16gb Macbook could run it (but not much else).
isn't it 2 bytes (fp16) per param. so 7b = 14 GB+some for inference?
it's very common to run local models in 8 bit int.
Re: Mistral NeMo
#107Did anyone try to check how are it's multilingual skills vs. Gemma 2? On the page, it's compared with LLama 3 only.
Well it's not on Le Chat, it's not on LMSys, it has a new tokenizer that breaks llama.cpp compatibility, and I'm sure as hell not gonna run it with Crapformers at 0.1x speed which as of right now seems to be the only way to actually test it out.
Re: Mistral NeMo
#108Pardon me if this is a dumb question, but is it possible for me to download these models into my computer (I have a 1080ti and a [2|3]070ti) and generate some sort of api interface? That way I can write programs that calls this API, and I find this appealing. EDIT: This a 1W light bulb moment for me, thank you!
I’d probably check https://ollama.com/library?q=Nemo in a couple of days. My guess is that by then ollama will have support for it. And you can then run the model locally on your machine with ollama.
We're working on it, except that there is a change to the tokenizer which we're still working through in our conversion scripts. Unfortunately we don't get a heads up from Mistral when they drop a model, so sometimes it takes a little bit of time to sort out the differences.
Also, I'm not sure if we'll call it mistral-nemo or nemo yet. :-D
Re: Mistral NeMo
#109Pardon me if this is a dumb question, but is it possible for me to download these models into my computer (I have a 1080ti and a [2|3]070ti) and generate some sort of api interface? That way I can write programs that calls this API, and I find this appealing. EDIT: This a 1W light bulb moment for me, thank you!
First thing I did when i saw the headline was to look for it on ollma but it didn't land there yet: https://ollama.com/library?sort=newest&q=NeMo
We're working on it!
Re: Mistral NeMo
#110Earlier quoted context omitted.
Generally yes, with caveats. There was some research showing that training a model on facts like "the mother of John Smith is Alice" but in German allowed it to answer questions like "who's the mother of John Smith", but not questions like "what's the name of Alice's child", regardless of language. Not sure if this holds at larger model sizes though, it's the sort of problem that's usually fixable by throwing more pa…
> Language models definitely do generalize to some extend and they're not "stochastic parrots" as previously thought, but there are some weird ways in which we expect them to generalize but they don't. Do you have any good sources that explain this? I was always thinking LLMs are indeed stochastic parrots, but language (that is the unified corpus of all languages in the training data) already inherently contains the…
> language already inherently contains the „generalization“
The mental gymnastics required to handwave language model capabilities are getting funnier and funnier every day.