Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Llama 3.2: Revolutionizing edge AI and vision with open, customizable models

ai.meta.com

Post reply on HN
Print view View original

21–30 of 343 posts

Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models

#21

Post by sva_ »

Curious about the multimodal model's architecture. But alas, when I try to request access

> Llama 3.2 Multimodal is not available in your region.

It sounds like they input the continuous output of an image encoder into a transformer, similar to transfusion[0]? Does someone know where to find more details?

Edit:

> Regarding the licensing terms, Llama 3.2 comes with a very similar license to Llama 3.1, with one key difference in the acceptable use policy: any individual domiciled in, or a company with a principal place of business in, the European Union is not being granted the license rights to use multimodal models included in Llama 3.2. [1]

What a bummer.

0. https://www.arxiv.org/abs/2408.11039

1. https://huggingface.co/blog/llama32#llama-32-license-changes...

Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models

#22

Post by [deleted] »

TheAceOfHearts wrote: post #16
paxys wrote: post #9

Earlier quoted context omitted.

Non US citizens can access the model just fine, if that's what you are implying.

I'm not implying anything. It's just frustrating that despite being a US territory with US citizens, PR isn't allowed to use this service without any explanation.

[deleted]

Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models

#23

Post by getcrunk »

Still no 14/30b parameter models since llama 2. Seriously killing real usability for power users/diy.

The 7/8B models are great for poc and moving to edge for minor use cases … but there’s a big and empty gap till 70b that most people can’t run.

The tin foil hat in me is saying this is the compromise the powers that be have agreed too. Basically being “open” but practically gimped for average joe techie. Basically arms control

Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models

#24

Post by kingkongjaffa »

llama3.2:3b-instruct-q8_0 is performing better than 3.1 8b-q4 on my macbookpro M1. It's faster and the results are better. It answered a few riddles and thought experiments better despite being 3b vs 8b.

I just removed my install of 3.1-8b.

my ollama list is currently:

$ ollama list

NAME ID SIZE MODIFIED

llama3.2:3b-instruct-q8_0 e410b836fe61 3.4 GB 2 hours ago

gemma2:9b-instruct-q4_1 5bfc4cf059e2 6.0 GB 3 days ago

phi3.5:3.8b-mini-instruct-q8_0 8b50e8e1e216 4.1 GB 3 days ago

mxbai-embed-large:latest 468836162de7 669 MB 3 months ago

Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models

#25

Post by paxys »

TheAceOfHearts wrote: post #16
paxys wrote: post #9

Earlier quoted context omitted.

Non US citizens can access the model just fine, if that's what you are implying.

I'm not implying anything. It's just frustrating that despite being a US territory with US citizens, PR isn't allowed to use this service without any explanation.

Just because you cannot access the model doesn't mean all of Puerto Rico is blocked.

Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models

#26

Post by _ink_ »

sva_ wrote: post #21

Curious about the multimodal model's architecture. But alas, when I try to request access > Llama 3.2 Multimodal is not available in your region. It sounds like they input the continuous output of an image encoder into a transformer, similar to transfusion[0]? Does someone know where to find more details? Edit: > Regarding the licensing terms, Llama 3.2 comes with a very similar license to Llama 3.1, with one key dif…

Oh. That's sad indeed. What might be the reason for excluding Europe?

Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models

#28

Post by swader999 »

getcrunk wrote: post #23

Still no 14/30b parameter models since llama 2. Seriously killing real usability for power users/diy. The 7/8B models are great for poc and moving to edge for minor use cases … but there’s a big and empty gap till 70b that most people can’t run. The tin foil hat in me is saying this is the compromise the powers that be have agreed too. Basically being “open” but practically gimped for average joe techie. Basically ar…

You don't need an F-15 to play at least, a decent sniper rifle will do. You can still practise even with a pellet gun. I'm running 70b models on my M2 max with 96 ram. Even larger models sort of work, although I haven't really put much time into anything above 70b.

Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models

#29

Post by Arubis »

_ink_ wrote: post #26
sva_ wrote: post #21

Curious about the multimodal model's architecture. But alas, when I try to request access > Llama 3.2 Multimodal is not available in your region. It sounds like they input the continuous output of an image encoder into a transformer, similar to transfusion[0]? Does someone know where to find more details? Edit: > Regarding the licensing terms, Llama 3.2 comes with a very similar license to Llama 3.1, with one key dif…

Oh. That's sad indeed. What might be the reason for excluding Europe?

Glibly, Europe has the gall to even consider writing regulations without asking the regulated parties for permission.

Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models

#30

Post by GaggiX »

sva_ wrote: post #21

Curious about the multimodal model's architecture. But alas, when I try to request access > Llama 3.2 Multimodal is not available in your region. It sounds like they input the continuous output of an image encoder into a transformer, similar to transfusion[0]? Does someone know where to find more details? Edit: > Regarding the licensing terms, Llama 3.2 comes with a very similar license to Llama 3.1, with one key dif…

Fortunately, Qwen-2-VL exists, it is pretty good and under an actual open source license, Apache 2.0.

Edit: the larger 72B model is not under Apache 2.0 but https://huggingface.co/Qwen/Qwen2-VL-72B-Instruct/blob/main/...

Qwen2-VL-72B seems to perform better than llama-3.2-90B on visual tasks.

Post reply on HN