Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Llama 3.2: Revolutionizing edge AI and vision with open, customizable models

ai.meta.com

Post reply on HN
Print view View original

311–320 of 343 posts

Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models

#311

Post by asterix_pano »

opdahl wrote: post #6

I'm blown away with just how open the Llama team at Meta is. It is nice to see that they are not only giving access to the models, but they at the same time are open about how they built them. I don't know how the future is going to go in the terms of models, but I sure am grateful that Meta has taken this position, and are pushing more openness.

Maybe it's cynical to think that way but maybe it's a way to crush the competition before it even begins: I would probably not invest in researching LLMs now, knowing that there is a company that will very likely produce a model close enough for free and I will likely never make back the investment.

Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models

#312

Post by grahamj »

edc117 wrote: post #291
grahamj wrote: post #279

Earlier quoted context omitted.

God isn’t real and I don’t want any person - or computer - to believe otherwise.

If you don't have any proof of that, you're no different than those that believe he exists. (Respectfully) Agnosticism really is the only correct scientific approach.

I have to disagree with that. Yes, ideally we should only believe things for which there is proof, but that is simply not an option for a great many things in our lives and the universe.

A lot of the time we have to fall back to estimating how plausible something is based on the knowledge we do have. Even in science it’s common for outcomes to be probabilistic rather than absolute.

So I say there is no god because, to my mind, the claim makes no sense. There is nothing I have ever seen, or that science has ever collected data on, to indicate that such a thing is plausible. It’s a myth, a fairy tale. I don’t need to prove otherwise because the onus of proof is on the one making the incredible claim.

Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models

#313

Post by grahamj »

altruios wrote: post #300
grahamj wrote: post #279

Earlier quoted context omitted.

God isn’t real and I don’t want any person - or computer - to believe otherwise.

God is not physically real. Neither are numbers. Both come from thinking minds. God is an egregore. It may be useful to model the various religions as singular entities under this lens, not true in the strictest sense, but useful none the less. God, Santa, and (our {human} version of) Math: all exist in 'mental space', they are models of the world (one is a significantly more accurate model, obviously). Atheist here:…

I could get behind that but people that believe in god tend to think of it as a real, physical (or at least metaphysical) thing.

For my own sanity I try to think of those who believe in literal god as simply confusing it with the universe itself. The universe created us, it nurtures us, it’s sort of timeless and immortal. If only they could just leave it at that.

Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models

#314

Post by sgt »

grahamj wrote: post #308
sgt wrote: post #189

Anyone on HN running models on their own local machines, like smaller Llama models or such? Or something else?

Doesn’t everyone? X) it’s super easy now with ollama + openwebui or an all in 1 like mlstudio

Was just concerned I don't have enough RAM. I have 16GB (M2 Pro). Got amazing mem bandwidth though (800GB/s)

Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models

#315

Post by jona-f »

Eisenstein wrote: post #256
jona-f wrote: post #215

Earlier quoted context omitted.

Should the line "p.o. 5rd w/ new W5 533" say "p.o. 3rd w/ new WW 5W .533R"? What does p.o. stand for? I can't make out the first letter. It looks more like the f, but the nodge on the upper left only fits the p. All the other p's look very different though.

'Replaced R436, R430 emitter resistors on right-channel power output board with new wire-wound 5watt .33ohm 5% with ceramic lead insulators'

Thx :). I thought the 3 looked like a b but didn't think brd would make any sense. My reasoning has led me astray.

Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models

#316

Post by svieira »

grahamj wrote: post #312
edc117 wrote: post #291

Earlier quoted context omitted.

If you don't have any proof of that, you're no different than those that believe he exists. (Respectfully) Agnosticism really is the only correct scientific approach.

I have to disagree with that. Yes, ideally we should only believe things for which there is proof, but that is simply not an option for a great many things in our lives and the universe. A lot of the time we have to fall back to estimating how plausible something is based on the knowledge we do have. Even in science it’s common for outcomes to be probabilistic rather than absolute. So I say there is no god because, t…

> There is nothing I have ever seen, or that science has ever collected data on, to indicate that such a thing is plausible.

Given that this is an estimate could you estimate what kind of thing you would have to see or what shape of data collected by science that would make you reconsider the plausibility of the existence of a supreme being?

Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models

#317

Post by kingkongjaffa »

aryehof wrote: post #162
kingkongjaffa wrote: post #24

llama3.2:3b-instruct-q8_0 is performing better than 3.1 8b-q4 on my macbookpro M1. It's faster and the results are better. It answered a few riddles and thought experiments better despite being 3b vs 8b. I just removed my install of 3.1-8b. my ollama list is currently: $ ollama list NAME ID SIZE MODIFIED llama3.2:3b-instruct-q8_0 e410b836fe61 3.4 GB 2 hours ago gemma2:9b-instruct-q4_1 5bfc4cf059e2 6.0 GB 3 days ago p…

On what basis do you use these different models?

mxbai is for embeddings for RAG.

The others are for text generation / instruction following, for various writing tasks.

Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models

#318

Post by Eisenstein »

jona-f wrote: post #315
Eisenstein wrote: post #256

Earlier quoted context omitted.

'Replaced R436, R430 emitter resistors on right-channel power output board with new wire-wound 5watt .33ohm 5% with ceramic lead insulators'

Thx :). I thought the 3 looked like a b but didn't think brd would make any sense. My reasoning has led me astray.

Yeah. If you realize that a large part of the llm's 'ocr' is guessing due to context (token prediction) and not actually recognizing the characters exactly, you can see that it is indeed pretty impressive because the log it is reading uses pretty unique terminology that it couldn't know from training.

Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models

#319

Post by snek_case »

asterix_pano wrote: post #311
opdahl wrote: post #6

I'm blown away with just how open the Llama team at Meta is. It is nice to see that they are not only giving access to the models, but they at the same time are open about how they built them. I don't know how the future is going to go in the terms of models, but I sure am grateful that Meta has taken this position, and are pushing more openness.

Maybe it's cynical to think that way but maybe it's a way to crush the competition before it even begins: I would probably not invest in researching LLMs now, knowing that there is a company that will very likely produce a model close enough for free and I will likely never make back the investment.

I don't think it's necessarily the small competitors that they are worried about, but they could be trying to prevent OpenAI from becoming too powerful and competing with them.

Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models

#320

Post by kristianp »

regularfry wrote: post #184
foxhop wrote: post #70

Earlier quoted context omitted.

4090 has 24G So we really need ~40B or G model (two cards) or like a ~20B with some room for context window. 5090 has ??G - still unreleased

Qwen2.5 has a 32B release, and quantised at q5_k_m it *just about" completely fills a 4090. It's a good model, too.

Do you also need space for context on the card to get decent speed though?
Post reply on HN