Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Llama 3.2: Revolutionizing edge AI and vision with open, customizable models

ai.meta.com

Post reply on HN
Print view View original

191–200 of 343 posts

Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models

#191

Post by troupo »

warkdarrior wrote: post #65
nickpsecurity wrote: post #10

Earlier quoted context omitted.

Do they tell you what training data they use for alignment? As in, what biases they intentionally put in the system they’re widely deploying?

Do you have some concrete example of biases in their models? Or are you just fishing for something to complain about?

The concrete example is that Meta opted everyone on their platform by default into providing content for their models without any consent.

The source and the quality of training data is important without looking for specific examples of a bias.

Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models

#192

Post by jsarv »

Patrick_Devine wrote: post #37
moffkalast wrote: post #13

I've just tested the 1B and 3B at Q8, some interesting bits: - The 1B is extremely coherent (feels something like maybe Mistral 7B at 4 bits), and with flash attention and 4 bit KV cache it only uses about 4.2 GB of VRAM for 128k context - A Pi 5 runs the 1B at 8.4 tok/s, haven't tested the 3B yet but it might need a lower quant to fit it and with 9T training tokens it'll probably degrade pretty badly - The 3B is a c…

Hoping to get this out soon w/ Ollama. Just working out a couple of last kinks. The 11b model is legit good though, particularly for tasks like OCR. It can actually read my cursive handwriting.

Naah, Qwen2-VL-7b still is much much better than 11b model for handwritten OCR from what i have tested. The 11b model hallucinates in case of handwritten OCR.

Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models

#193

Post by khafra »

kgeist wrote: post #186

Tried the 1B model with the "think step by step" prompt. It gets "which is larger: 9.11 or 9.9?" right if it manages to mention that decimals need to be compared first in its step-by-step thinking. If it skips mentioning decimals, then it says 9.11 is larger. It gets the strawberry question wrong even after enumerating all the letters correctly, probably because it can't properly count.

Of course, in many contexts, it is correct to put 9.11 after 9.9--software versioning does it that way, for example.

Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models

#194

Post by isoprophlex »

opdahl wrote: post #6

I'm blown away with just how open the Llama team at Meta is. It is nice to see that they are not only giving access to the models, but they at the same time are open about how they built them. I don't know how the future is going to go in the terms of models, but I sure am grateful that Meta has taken this position, and are pushing more openness.

They're out to fuck over the competition by killing their moat. Classic commoditize your complement.

Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models

#195

Post by mike_hearn »

nickpsecurity wrote: post #122
warkdarrior wrote: post #65

Earlier quoted context omitted.

Do you have some concrete example of biases in their models? Or are you just fishing for something to complain about?

Google’s and OpenAI often answered far-left, Progressive, and atheist. Google’s was censoring white people at one point. Facebook seems to espouse similar values. They’ve funded work to increase those values. Many mention topics relevant to these things in their papers in the bias or alignment sections. These political systems don’t represent the majority of the world. They might not even represent half the U.S.. Peo…

You're objectively correct but judging from your downvotes there seems to be some denial here about that! The atheism alone means it's different from a big chunk of the world's population, possibly the majority. Supposedly around 80% of the world's population identify with a religion though I guess you can debate how many people are truly devout.

The good news is that the big AI labs seem to be slowly getting a grip on the misalignment of their safety teams. If you look at the extensive docs Meta provide for this model they do talk about safety training, and it's finally of the reasonable and non-ideological kind. They're trying to stop it from hacking computers, telling people how to build advanced weaponry and so on. There are valid use cases for all of those things, and you could argue there's no point when the knowledge came from books+internet to begin with, but everyone can agree that there are at least genuine safety-related issues with those topics.

The possible exception here is Google. They seem to be the worst affected of all the big labs.

Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models

#196

Post by cedws »

opdahl wrote: post #6

I'm blown away with just how open the Llama team at Meta is. It is nice to see that they are not only giving access to the models, but they at the same time are open about how they built them. I don't know how the future is going to go in the terms of models, but I sure am grateful that Meta has taken this position, and are pushing more openness.

Zuckerberg probably realises the value of currying favour with engineers. Also, I think he has a personal vendetta to compete with Musk in this space.

Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models

#197

Post by weberer »

_ink_ wrote: post #26
sva_ wrote: post #21

Curious about the multimodal model's architecture. But alas, when I try to request access > Llama 3.2 Multimodal is not available in your region. It sounds like they input the continuous output of an image encoder into a transformer, similar to transfusion[0]? Does someone know where to find more details? Edit: > Regarding the licensing terms, Llama 3.2 comes with a very similar license to Llama 3.1, with one key dif…

Oh. That's sad indeed. What might be the reason for excluding Europe?

According to the open letter they linked, it looks to be regarding some regulation about the training data used.

https://euneedsai.com/

Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models

#198

Post by vorticalbox »

calgoo wrote: post #170
refulgentis wrote: post #127

Earlier quoted context omitted.

ollama is "just" llama.cpp underneath, I recommend switching to LM Studio or Jan, they don't have this issue of proprietary wrapper that obfuscates, you can just use any ol GGUF

I use gguf in ollama on a daily basis, so not sure what the issue is? Just wrap it in a modelfile and done!

I think because the larger models support images.

Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models

#199

Post by seydor »

isoprophlex wrote: post #194
opdahl wrote: post #6

I'm blown away with just how open the Llama team at Meta is. It is nice to see that they are not only giving access to the models, but they at the same time are open about how they built them. I don't know how the future is going to go in the terms of models, but I sure am grateful that Meta has taken this position, and are pushing more openness.

They're out to fuck over the competition by killing their moat. Classic commoditize your complement.

I believe the most important contribution is to show that super-funded companies don't really have a special moat: Llama is transformers, they just have the money to scale it. Many entities around the world can replicate this and it seems Meta is doing it before they do.

Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models

#200

Post by sva_ »

ankit219 wrote: post #54
sva_ wrote: post #21

Curious about the multimodal model's architecture. But alas, when I try to request access > Llama 3.2 Multimodal is not available in your region. It sounds like they input the continuous output of an image encoder into a transformer, similar to transfusion[0]? Does someone know where to find more details? Edit: > Regarding the licensing terms, Llama 3.2 comes with a very similar license to Llama 3.1, with one key dif…

If you are still curious about the architecture, from the blog: > To add image input support, we trained a set of adapter weights that integrate the pre-trained image encoder into the pre-trained language model. The adapter consists of a series of cross-attention layers that feed image encoder representations into the language model. We trained the adapter on text-image pairs to align the image representations with t…

Thanks for the info, I now also found the model card. So it seems like they went the way of grafting models together, which I find less interesting tbh.

In the Transfusion paper, they use both discrete (text tokens) and continuous (images) signals to train a single transformer. To do this, they use a VAE to create a latent representation of the images (split into patches) which are fed into the transformer within one linear sequence along the text tokens - they trained the whole model from scratch (the largest being a 7B model trained on 2T token with a 1:1 split text:images.) The loss they trained the model on was a combination of the normal language modeling LM loss (cross entropy on tokens) and diffusion DDPM on the images.

There was some prior art on this, but models like Chameleon discretized the images into a token codebook of a certain size - so there were special tokens representing the images. However, this incurred a severe information loss which Transfusion claims to have alleviated using the continuous latent vectors of images.

Training a single set of weights (shared weights) on different modalities seems more interesting looking forward, in particular for emergent phenomena imo.

Some of the authors of the transfusion paper work at meta so I was hoping they trained a larger-scale model. Or released any transfusion-based weights at all.

Anyways, exciting stuff either ways.

Post reply on HN