Tried out 3B on ollama, asking questions in optics, bio, and rust. It's super fast with a lot of knowledge, a large context and great understanding. Really impressive model.
I question whether a 3B model can have “a lot of knowledge”.
Llama 3.2: Revolutionizing edge AI and vision with open, customizable models
51–60 of 343 posts
Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models
#52What mobile devices can the smaller models run on? iPhone, Android?
Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models
#53Curious about the multimodal model's architecture. But alas, when I try to request access > Llama 3.2 Multimodal is not available in your region. It sounds like they input the continuous output of an image encoder into a transformer, similar to transfusion[0]? Does someone know where to find more details? Edit: > Regarding the licensing terms, Llama 3.2 comes with a very similar license to Llama 3.1, with one key dif…
I hereby grant license to anyone in the EU to do whatever they want with this.
Cheers :)
Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models
#54Curious about the multimodal model's architecture. But alas, when I try to request access > Llama 3.2 Multimodal is not available in your region. It sounds like they input the continuous output of an image encoder into a transformer, similar to transfusion[0]? Does someone know where to find more details? Edit: > Regarding the licensing terms, Llama 3.2 comes with a very similar license to Llama 3.1, with one key dif…
If you are still curious about the architecture, from the blog:
> To add image input support, we trained a set of adapter weights that integrate the pre-trained image encoder into the pre-trained language model. The adapter consists of a series of cross-attention layers that feed image encoder representations into the language model. We trained the adapter on text-image pairs to align the image representations with the language representations. During adapter training, we also updated the parameters of the image encoder, but intentionally did not update the language-model parameters. By doing that, we keep all the text-only capabilities intact, providing developers a drop-in replacement for Llama 3.1 models.
What this crudely means is that they extended the base Llama 3.1, to include image based weights and inference. You can do that if you freeze the existing weights. add new ones which are then updated during training runs (adapter training). Then they did SFT and RLHF runs on the composite model (for lack of a better word). This is a little known technique, and very effective. I just had a paper accepted about a similar technique, will share a blog once that is published if you are interested (though it's not on this scale, and probably not as effective). Side note: That is also why you see param size of 11B and 90B as addition from the text only models.
Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models
#55I'm absolutely amazed at how capable the new 1B model is, considering it's just a 1.3GB download (for the Ollama GGUF version).
I tried running a full codebase through it (since it can handle 128,000 tokens) and asking it to summarize the code - it did a surprisingly decent job, incomplete but still unbelievable for a model that tiny: https://gist.github.com/simonw/64c5f5b111fe473999144932bef42...
More of my notes here: https://simonwillison.net/2024/Sep/25/llama-32/
I've been trying out the larger image models to using the versions hosted on https://lmarena.ai/ - navigate to "Direct Chat" and you can select them from the dropdown and upload images to run prompts.
Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models
#56I still can't access the hosted model at meta.ai from Puerto Rico, despite us being U.S. citizens. I don't know what Meta has against us. Could someone try giving the 90b model this word search problem [0] and tell me how it performs? So far with every model I've tried, none has ever managed to find a single word correctly. [0] https://imgur.com/i9Ps1v6
This is likely because the models use OCR on images with text, and once parsed the word search doesn't make sense anymore. Would be interesting to see a model just working on raw input though.
Image models such as Llama 3.2 11B and 90B (and the Claude 3 series, and Microsoft Phi-3.5-vision-instruct, and PaliGemma, and GPT-4o) don't run OCR as a separate step. Everything they do is from that raw vision model.
Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models
#57Curious about the multimodal model's architecture. But alas, when I try to request access > Llama 3.2 Multimodal is not available in your region. It sounds like they input the continuous output of an image encoder into a transformer, similar to transfusion[0]? Does someone know where to find more details? Edit: > Regarding the licensing terms, Llama 3.2 comes with a very similar license to Llama 3.1, with one key dif…
[dead]
Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models
#58Curious about the multimodal model's architecture. But alas, when I try to request access > Llama 3.2 Multimodal is not available in your region. It sounds like they input the continuous output of an image encoder into a transformer, similar to transfusion[0]? Does someone know where to find more details? Edit: > Regarding the licensing terms, Llama 3.2 comes with a very similar license to Llama 3.1, with one key dif…
Oh. That's sad indeed. What might be the reason for excluding Europe?
Punishment. "Your government passes laws we don't like, so we aren't going to let you have our latest toys".
Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models
#59Earlier quoted context omitted.
When I visit meta.ai it says: > Meta AI isn't available yet in your country Maybe it's just my ISP, I'll ask some friends if they can access the service.
meta.ai is their AI service (similar to ChatGPT). The model source itself is hosted on llama.com.
I'm aware. I wanted to try out their hosted version of the model because I'm GPU poor.
Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models
#60Curious about the multimodal model's architecture. But alas, when I try to request access > Llama 3.2 Multimodal is not available in your region. It sounds like they input the continuous output of an image encoder into a transformer, similar to transfusion[0]? Does someone know where to find more details? Edit: > Regarding the licensing terms, Llama 3.2 comes with a very similar license to Llama 3.1, with one key dif…
I hereby grant license to anyone in the EU to do whatever they want with this.
Well you said hereby so it must be law.