Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Llama 3.2: Revolutionizing edge AI and vision with open, customizable models

ai.meta.com

Post reply on HN
Print view View original

161–170 of 343 posts

Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models

#161

Post by littlestymaar »

imiric wrote: post #64
pocketarc wrote: post #35

Earlier quoted context omitted.

Between this and Apple's policies, big tech corporations really seem to be putting the screws to the EU as much as they can. "See, consumers? Look at how bad your regulation is, that you're missing out on all these cool things we're working on. Talk to your politicians!" Regardless of your political opinion on the subject, you've got to admit, at the very least, it will be educational to see how this develops over th…

> Between this and Apple's policies, big tech corporations really seem to be putting the screws to the EU as much as they can. Funny, I see that the other way around, actually. The EU is forcing Big Tech to be transparent and not exploit their users. It's the companies that must choose to comply, or take their business elsewhere. Let's not forget that Apple users in the EU can use 3rd-party stores, and it was EU regu…

> The reason Meta is not making their models available in the EU is because they can't or won't comply with the recent AI regulations. This only means that the law is working as intended.

It isn't clear at all, and in fact given how light handed the European Commission when dealing with infringement cases (no fine before lots of warning and even clarification meetings about how to comply with the law) Meta would take no risk at all releasing something now even if they needed to roll it back later.

They are definitely trying to put pressure on the European Commission, leveraging the fact that Thierry Breton was dismissed.

Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models

#162

Post by aryehof »

kingkongjaffa wrote: post #24

llama3.2:3b-instruct-q8_0 is performing better than 3.1 8b-q4 on my macbookpro M1. It's faster and the results are better. It answered a few riddles and thought experiments better despite being 3b vs 8b. I just removed my install of 3.1-8b. my ollama list is currently: $ ollama list NAME ID SIZE MODIFIED llama3.2:3b-instruct-q8_0 e410b836fe61 3.4 GB 2 hours ago gemma2:9b-instruct-q4_1 5bfc4cf059e2 6.0 GB 3 days ago p…

On what basis do you use these different models?

Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models

#163

Post by littlestymaar »

sva_ wrote: post #92
littlestymaar wrote: post #85

Earlier quoted context omitted.

GP is talking about context windows, not the number of token used by the tokenizer.

Somewhat confusingly, it appears the tokenizer vocabulary as well as the context length are both 128k tokens!

Yup, that's why I wanted to clarify things.

Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models

#164

Post by doubtfuluser »

talldayo wrote: post #116
thefourthchime wrote: post #112

Earlier quoted context omitted.

They have a hose of ad money and have nothing to lose doing this. You can’t say that for the other guys.

I can absolutely say that about Google and Apple.

For Apple - maybe, but they also recently open sourced some of their models. For Google: they host and want to make money on the models by you using them on their platform.

Meta has no interest in that but directly benefits from advancements on top of Llama.

Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models

#165

Post by Patrick_Devine »

v3ss0n wrote: post #145
simonw wrote: post #139

Earlier quoted context omitted.

Not that I’ve seen so far, but Ollama are pending a solution for that “soon”.

I doubt ollama team can do much about it. Ollama are just wrapper on top of heavy lifter

The draft PRs are already up in the repo.

Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models

#166

Post by Patrick_Devine »

PhilippGille wrote: post #160
kingkongjaffa wrote: post #24

llama3.2:3b-instruct-q8_0 is performing better than 3.1 8b-q4 on my macbookpro M1. It's faster and the results are better. It answered a few riddles and thought experiments better despite being 3b vs 8b. I just removed my install of 3.1-8b. my ollama list is currently: $ ollama list NAME ID SIZE MODIFIED llama3.2:3b-instruct-q8_0 e410b836fe61 3.4 GB 2 hours ago gemma2:9b-instruct-q4_1 5bfc4cf059e2 6.0 GB 3 days ago p…

Aren't the _0 quantizations considered deprecated and _K_S or _K_M preferable? https://github.com/ollama/ollama/issues/5425

For _K_S definitely not. We quantized 3b with q4_K_M since we were getting good results out of it. Officially Meta has only talked about quantization for 405b and hasn't given any actual guidance for what the "best" quantization should be for the smaller models. With The 1b model we didn't see good results with any of the 4b quantizations and went with q8_0 as the default.

Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models

#167

Post by semi-extrinsic »

slt2021 wrote: post #147
ericjmorey wrote: post #93

Earlier quoted context omitted.

Even without intentionally biasing the model, without knowing the biases that exist in the training data, they're just biased black boxes that come with the overhead of figuring out how it's biased. All data is biased, there's no avoiding that fact.

bias is some normative lens that some people came up with, but it is purely subjective and is a social construct, that has roots in the area of social justice and has nothing to do with the LLM. the proof is that all critics of AI/LLM have never ever produced a single "unbiased" model. If unbiased model does not exist (at least I never seen an AI/LLM sceptics community produce one), then the concept of bias is useles…

If you forget about the social justice stuff for a minute, there are many other types of bias relevant for an LLM.

One example is US-centric bias. If I ask the LLM a question where the answer is one thing in the US and another thing in Germany, you can't really de-bias the model. But ideally you can have it request more details in order to give a good answer.

Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models

#169

Post by Tepix »

oulipo wrote: post #86

Can the 3B run on a M1 macbook? It seems that it hogs all the memory. The 1B runs fine

It all depends on the amount of RAM. Since you're not mentioning it, you probably have only the default 8GB? Still, the 3B should run even on an 8GB machine.

Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models

#170

Post by calgoo »

refulgentis wrote: post #127
freedomben wrote: post #114

If anyone else is looking for the bigger models on ollama and wondering where they are, the Ollama blog post answered that for me. The are "coming soon" so they just aren't ready quite yet[1]. I was a little worried when I couldn't find them but sounds like we just need to be patient. [1]: https://ollama.com/blog/llama3.2

ollama is "just" llama.cpp underneath, I recommend switching to LM Studio or Jan, they don't have this issue of proprietary wrapper that obfuscates, you can just use any ol GGUF

I use gguf in ollama on a daily basis, so not sure what the issue is? Just wrap it in a modelfile and done!
Post reply on HN