41–50 of 165 posts
I wonder why Mistral et al don't prepare GGUF versions of these for launch day? If I were them I'd want to be the default source of the versions of my models that people use, rather than farming that out to whichever third party races to publish the GGUF (and other formats) first.
Depends on which community you are in as to what you want.
I find it interesting how coding/software development still appears to be the one category that these most popular models release specialised models for. Where's the finance or legal models from Mistral or Meta or OpenAI? Perhaps it's just confirmation bias, but programming really does seem to be the ideal usecase for LLMs in a way that other professions just haven't been able to crack. Compared to other types of wor…
The explanation is easier, I think. Consider what data these models are trained on, and who are the immediate developers of these models. The models are trained on a vast set of whatever is available on the internet. They are developed by tech people/programmers who are surprisingly blind to their own biases and interests. There's no surprise that one of the main things they want to try and do is programming, using v…
Section 230.
It's been argued that a response by a LLM, to user input, is "user-generated content" and hence the platform has generally no liability (except CSAM).
Nobody has successfully sued.
I’m AI stupid. Does anyone know if training on multiple languages provides “cross-over” — so training done in German can be utilized when answering a prompt in English? I once went through various Wikipedia articles in a couple languages and the differences were interesting. For some reason I thought they’d be almost verbatim (forgetting that’s not how Wikipedia works!) and while I can’t remember exactly I felt they…
no, it is basically an 'auto-correct' spell checker from the phone. It only knows what it was trained on. But it has been shown that a coding LLM that has never seen a programming language or a library can "learn" a new one faster than, say, a generic LLM.
IE you train an LLM on both English and French in general, but only teach it a specific fact in French, it can give you that fact in English
> Mistral NeMo comes packaged as an NVIDIA NIM inference microservice, offering performance-optimized inference with NVIDIA TensorRT-LLM engines.
> *Designed to fit on the memory of a single NVIDIA L40S, NVIDIA GeForce RTX 4090 or NVIDIA RTX 4500 GPU*, the Mistral NeMo NIM offers high efficiency, low compute cost, and enhanced security and privacy.
> The model was trained using Megatron-LM, part of NVIDIA NeMo, with 3,072 H100 80GB Tensor Core GPUs on DGX Cloud, composed of NVIDIA AI architecture, including accelerated computing, network fabric and software to increase training efficiency.
Is "Parameter Creep" going to becomes a thing? They hold up Llama-8b as a competitor despite NeMo having 50% more parameters. The same thing happened with gemma-27b, where they compared it to all the 7-9b models. It seems like an easy way to boost benchmarks while coming off as "small" at first glance.
- 3B for CPU inference or running on edge devices.
- 20-30B for maximizing single consumer GPU potential.
- 70B+ for those who can afford it.
7-9B never felt like an ideal size.
Also, are these small models OSS? Easier self hosting seems to be the main benefo for small models.
I’m AI stupid. Does anyone know if training on multiple languages provides “cross-over” — so training done in German can be utilized when answering a prompt in English? I once went through various Wikipedia articles in a couple languages and the differences were interesting. For some reason I thought they’d be almost verbatim (forgetting that’s not how Wikipedia works!) and while I can’t remember exactly I felt they…
Performance improved across all benchmarks; in English (the original language).
I’m AI stupid. Does anyone know if training on multiple languages provides “cross-over” — so training done in German can be utilized when answering a prompt in English? I once went through various Wikipedia articles in a couple languages and the differences were interesting. For some reason I thought they’d be almost verbatim (forgetting that’s not how Wikipedia works!) and while I can’t remember exactly I felt they…
https://openreview.net/forum?id=KIPJKST4gw
Is symbolic language a fuzzy sort of code? Absolutely, because it conveys logic and information. TLDR: yes!
Is "Parameter Creep" going to becomes a thing? They hold up Llama-8b as a competitor despite NeMo having 50% more parameters. The same thing happened with gemma-27b, where they compared it to all the 7-9b models. It seems like an easy way to boost benchmarks while coming off as "small" at first glance.
Pardon me if this is a dumb question, but is it possible for me to download these models into my computer (I have a 1080ti and a [2|3]070ti) and generate some sort of api interface? That way I can write programs that calls this API, and I find this appealing. EDIT: This a 1W light bulb moment for me, thank you!
I think it should also run well on a 36GB MacBook Pro or probably a 24GB Macbook Air