Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Llama 3.2: Revolutionizing edge AI and vision with open, customizable models

ai.meta.com

Post reply on HN
Print view View original

61–70 of 343 posts

Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models

#61

Post by GaggiX »

simonw wrote: post #55

I'm absolutely amazed at how capable the new 1B model is, considering it's just a 1.3GB download (for the Ollama GGUF version). I tried running a full codebase through it (since it can handle 128,000 tokens) and asking it to summarize the code - it did a surprisingly decent job, incomplete but still unbelievable for a model that tiny: https://gist.github.com/simonw/64c5f5b111fe473999144932bef42... More of my notes he…

Llama 3.2 vision models don't seem that great if they have to compare them to Claude 3 Haiku or GPT4o-mini. For an open alternative I would use Qwen-2-72B model, it's smaller than the 90B and seems to perform quite better. Also Qwen2-VL-7B as an alternative to Llama-3.2-11B, smaller, better in visual benchmarks and also Apache 2.0.

Molmo models: https://huggingface.co/collections/allenai/molmo-66f379e6fe3..., also seem to perform better than Llama-3.2 models while being smaller and Apache 2.0.

Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models

#62

Post by DannyBee »

pocketarc wrote: post #35
Arubis wrote: post #29

Earlier quoted context omitted.

Glibly, Europe has the gall to even consider writing regulations without asking the regulated parties for permission.

Between this and Apple's policies, big tech corporations really seem to be putting the screws to the EU as much as they can. "See, consumers? Look at how bad your regulation is, that you're missing out on all these cool things we're working on. Talk to your politicians!" Regardless of your political opinion on the subject, you've got to admit, at the very least, it will be educational to see how this develops over th…

Or, again, they are just deciding the economy isn't worth the cost. (or not worth prioritizing upfront or ....)

When we had numerous discussions on HN as these rules were implemented, this is precisely what the europeans said should happen.

So why does it now have to be some concerted effort to "put the screws to EU"?

I otherwise agree it will be interesting, but mostly in the sense that i watched people swear up and down this was just about protecting EU citizens and they were fine with none of these companies doing anything in the EU or not prioritizing the EU if they decided it wasn't worth the cost.

We'll see if that's true or not, i guess, or if they really wanted it to be "you have to do it, but on our terms" or whatever.

Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models

#63

Post by [deleted] »

minimaxir wrote: post #18

Off topic/meta, but the Llama 3.2 news topic received many, many HN submissions and upvotes but never made it to the front page: the fact that it's on the front page now indicates that moderators intervened to rescue it: https://news.ycombinator.com/from?site=meta.com (showdead on) If there's an algorithmic penalty against the news for whatever reason, that may be a flaw in the HN ranking algorithm.

[deleted]

Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models

#64

Post by imiric »

pocketarc wrote: post #35
Arubis wrote: post #29

Earlier quoted context omitted.

Glibly, Europe has the gall to even consider writing regulations without asking the regulated parties for permission.

Between this and Apple's policies, big tech corporations really seem to be putting the screws to the EU as much as they can. "See, consumers? Look at how bad your regulation is, that you're missing out on all these cool things we're working on. Talk to your politicians!" Regardless of your political opinion on the subject, you've got to admit, at the very least, it will be educational to see how this develops over th…

> Between this and Apple's policies, big tech corporations really seem to be putting the screws to the EU as much as they can.

Funny, I see that the other way around, actually. The EU is forcing Big Tech to be transparent and not exploit their users. It's the companies that must choose to comply, or take their business elsewhere. Let's not forget that Apple users in the EU can use 3rd-party stores, and it was EU regulations that forced Apple to switch to USB-C. All of these are a win for consumers.

The reason Meta is not making their models available in the EU is because they can't or won't comply with the recent AI regulations. This only means that the law is working as intended.

> it will be educational to see how this develops over the next 5-10 years of tech progress, as the EU gets excluded from more and more things.

I don't think we're missing much that Big Tech has to offer, and we'll probably be better off for it. I'm actually in favor of even stricter regulations, particularly around AI, but what was recently enacted is a good start.

Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models

#65

Post by warkdarrior »

nickpsecurity wrote: post #10
opdahl wrote: post #6

I'm blown away with just how open the Llama team at Meta is. It is nice to see that they are not only giving access to the models, but they at the same time are open about how they built them. I don't know how the future is going to go in the terms of models, but I sure am grateful that Meta has taken this position, and are pushing more openness.

Do they tell you what training data they use for alignment? As in, what biases they intentionally put in the system they’re widely deploying?

Do you have some concrete example of biases in their models? Or are you just fishing for something to complain about?

Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models

#66

Post by [deleted] »

sva_ wrote: post #21

Curious about the multimodal model's architecture. But alas, when I try to request access > Llama 3.2 Multimodal is not available in your region. It sounds like they input the continuous output of an image encoder into a transformer, similar to transfusion[0]? Does someone know where to find more details? Edit: > Regarding the licensing terms, Llama 3.2 comes with a very similar license to Llama 3.1, with one key dif…

[deleted]

Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models

#67

Post by [deleted] »

nmwnmw wrote: post #5

- Llama 3.2 introduces small vision LLMs (11B and 90B parameters) and lightweight text-only models (1B and 3B) for edge/mobile devices, with the smaller models supporting 128K token context. - The 11B and 90B vision models are competitive with leading closed models like Claude 3 Haiku on image understanding tasks, while being open and customizable. - Llama 3.2 comes with official Llama Stack distributions to simplify…

[deleted]

Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models

#69

Post by foxhop »

tomComb wrote: post #51
dhbradshaw wrote: post #19

Tried out 3B on ollama, asking questions in optics, bio, and rust. It's super fast with a lot of knowledge, a large context and great understanding. Really impressive model.

I question whether a 3B model can have “a lot of knowledge”.

My guess is it uses the same vocabulary size as llama 3.1 which is 128,000 different tokens (words) to support many languages. Parameter count is less of an indicator of fitness than previously thought.

Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models

#70

Post by foxhop »

getcrunk wrote: post #23

Still no 14/30b parameter models since llama 2. Seriously killing real usability for power users/diy. The 7/8B models are great for poc and moving to edge for minor use cases … but there’s a big and empty gap till 70b that most people can’t run. The tin foil hat in me is saying this is the compromise the powers that be have agreed too. Basically being “open” but practically gimped for average joe techie. Basically ar…

4090 has 24G

So we really need ~40B or G model (two cards) or like a ~20B with some room for context window.

5090 has ??G - still unreleased

Post reply on HN