Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Llama 3.2: Revolutionizing edge AI and vision with open, customizable models

ai.meta.com

Post reply on HN
Print view View original

71–80 of 343 posts

Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models

#71

Post by foxhop »

simonw wrote: post #55

I'm absolutely amazed at how capable the new 1B model is, considering it's just a 1.3GB download (for the Ollama GGUF version). I tried running a full codebase through it (since it can handle 128,000 tokens) and asking it to summarize the code - it did a surprisingly decent job, incomplete but still unbelievable for a model that tiny: https://gist.github.com/simonw/64c5f5b111fe473999144932bef42... More of my notes he…

The llama 3.0, 3.1, & 3.2 all use the TikToken tokenizer which is the open source openai tokenizer.

Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models

#72

Post by elcomet »

TheAceOfHearts wrote: post #59
paxys wrote: post #50

Earlier quoted context omitted.

meta.ai is their AI service (similar to ChatGPT). The model source itself is hosted on llama.com.

I'm aware. I wanted to try out their hosted version of the model because I'm GPU poor.

You can try it on hugging face

Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models

#73

Post by taneq »

kingkongjaffa wrote: post #24

llama3.2:3b-instruct-q8_0 is performing better than 3.1 8b-q4 on my macbookpro M1. It's faster and the results are better. It answered a few riddles and thought experiments better despite being 3b vs 8b. I just removed my install of 3.1-8b. my ollama list is currently: $ ollama list NAME ID SIZE MODIFIED llama3.2:3b-instruct-q8_0 e410b836fe61 3.4 GB 2 hours ago gemma2:9b-instruct-q4_1 5bfc4cf059e2 6.0 GB 3 days ago p…

For a second I read that as “it just removed my install of 3.1-8b” :D

Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models

#74

Post by gunalx »

3b was pretty good at multimodal (Norwegian) still a lot of gibberish at times, and way more sensitive than 8b but more usable than Gemma 2 2b at multi modal, fine at my python list sorter with args standard question. But 90b vision just refuses all my actually useful tasks like helping recreate the images in html or do anything useful with the image data other than describing it. Have not gotten as stuck with 70b or openai before. Insane amount of refusals all the time.

Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models

#76

Post by lolinder »

foxhop wrote: post #69
tomComb wrote: post #51

Earlier quoted context omitted.

I question whether a 3B model can have “a lot of knowledge”.

My guess is it uses the same vocabulary size as llama 3.1 which is 128,000 different tokens (words) to support many languages. Parameter count is less of an indicator of fitness than previously thought.

That doesn't address the thing they're skeptical about, which is how much knowledge can be encoded in 3B parameters.

3B models are great for text manipulation, but I've found them to be pretty bad at having a broad understanding of pragmatics or any given subject. The larger models encode a lot more than just language in those 70B+ parameters.

Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models

#77

Post by [deleted] »

foxhop wrote: post #70
getcrunk wrote: post #23

Still no 14/30b parameter models since llama 2. Seriously killing real usability for power users/diy. The 7/8B models are great for poc and moving to edge for minor use cases … but there’s a big and empty gap till 70b that most people can’t run. The tin foil hat in me is saying this is the compromise the powers that be have agreed too. Basically being “open” but practically gimped for average joe techie. Basically ar…

4090 has 24G So we really need ~40B or G model (two cards) or like a ~20B with some room for context window. 5090 has ??G - still unreleased

[deleted]

Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models

#78

Post by daemonologist »

TheAceOfHearts wrote: post #3

I still can't access the hosted model at meta.ai from Puerto Rico, despite us being U.S. citizens. I don't know what Meta has against us. Could someone try giving the 90b model this word search problem [0] and tell me how it performs? So far with every model I've tried, none has ever managed to find a single word correctly. [0] https://imgur.com/i9Ps1v6

Both Llama 3.2 90B and Claude 3.5 Sonnet can find "turkey" and "spoon", probably because they're left-to-right. Llama gave approximate locations for each and Claude gave precise but slightly incorrect locations. Further prompting to look for diagonal and right-to-left words returned plausible but incorrect responses, slightly more plausible from Claude than Llama. (In this test I cropped the word search to just the letter grid, and asked the model to find any English words related to soup.)

Anyways, I think there just isn't a lot of non-right-to-left English in the training data. A word search is pretty different from the usual completion, chat, and QA tasks these models are oriented towards; you might be able to get somewhere with fine-tuning though.

Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models

#79

Post by create-username »

dhbradshaw wrote: post #19

Tried out 3B on ollama, asking questions in optics, bio, and rust. It's super fast with a lot of knowledge, a large context and great understanding. Really impressive model.

Can it speak foreign languages like German, Spanish, Ancient Greek?

Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models

#80

Post by luke-stanley »

getcrunk wrote: post #23

Still no 14/30b parameter models since llama 2. Seriously killing real usability for power users/diy. The 7/8B models are great for poc and moving to edge for minor use cases … but there’s a big and empty gap till 70b that most people can’t run. The tin foil hat in me is saying this is the compromise the powers that be have agreed too. Basically being “open” but practically gimped for average joe techie. Basically ar…

The Llama 3.2 11B multimodal model is a bit less than 14B but smaller models can do more these days, and Meta are not the only ones making models. The 70B model has been pruned down by NVIDIA if I recall correctly. The 405B model also will be shrunk down and can presumably be used to strengthen smaller models. I'm not convinced by your shiny hat.
Post reply on HN