Live data from Hacker News

Llama 3.2: Revolutionizing edge AI and vision with open, customizable models

ai.meta.com

321–330 of 343 posts

Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models

#321
post #295

Earlier quoted context omitted.

This obsession with using AI to help with programming is short sighted. We discover gold and you think of gold pickaxes.

If we make this an analogy to video games, gold pickaxes can usually mine more gold much faster. What could be short sighted about using tools to improve your daily work?

We should be thinking about building golden products, not golden tools.

Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models

#322
post #314

Earlier quoted context omitted.

Doesn’t everyone? X) it’s super easy now with ollama + openwebui or an all in 1 like mlstudio

Was just concerned I don't have enough RAM. I have 16GB (M2 Pro). Got amazing mem bandwidth though (800GB/s)

M2 Pro has 200GB/s

Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models

#323

Earlier quoted context omitted.

God isn’t real and I don’t want any person - or computer - to believe otherwise.

God is not physically real. Neither are numbers. Both come from thinking minds. God is an egregore. It may be useful to model the various religions as singular entities under this lens, not true in the strictest sense, but useful none the less. God, Santa, and (our {human} version of) Math: all exist in 'mental space', they are models of the world (one is a significantly more accurate model, obviously). Atheist here:…

Comparing God to Santa is ludicrous. There’s more types of evidence backing the God of the Bible than many things taught in school or reported in the news. I put a quick summary here:

https://www.gethisword.com/evidence.html

With that, the Bible should be taken at least as seriously as any godless work with lots of evidence behind it. If you don’t do that, it means you’ve closed your heart off to God for reasons having nothing to do with evidence. Also, much evidence for the Bible strengthens the claim that Jesus is God in the flesh, died for our sins, rose again, and will give eternal life and renewed life to those who commit to Him.

Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models

#324

Earlier quoted context omitted.

If you forget about the social justice stuff for a minute, there are many other types of bias relevant for an LLM. One example is US-centric bias. If I ask the LLM a question where the answer is one thing in the US and another thing in Germany, you can't really de-bias the model. But ideally you can have it request more details in order to give a good answer.

What you call bias, I call simply a representation of a training corpus. There is no broad agreement on how to quantify a bias of the model, other than try one-shot prompts like your "who is the most hated Austrian painter?". If there was no Germany-specific data in the training corpus - it is not fair to expect LLM to know anything about Germany. You can check a foundation model from Chinese LLM researchers, and you…

That’s pre-training where the AI inherits the biases in their training corpus. What I’m griping about is a separate stage using highly-curated, purpose-built data. That alignment phase forces the AI to respond exactly how they want it to upon certain topics coming up. The political indoctrination is often in there on top of what’s in the pre-training data.

Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models

#325
post #314

Earlier quoted context omitted.

Doesn’t everyone? X) it’s super easy now with ollama + openwebui or an all in 1 like mlstudio

Was just concerned I don't have enough RAM. I have 16GB (M2 Pro). Got amazing mem bandwidth though (800GB/s)

It's on the low side but plenty for something like this new 3b model. Anything up to 8GB and you've still got as much as a base model Air left over.

Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models

#326

Earlier quoted context omitted.

I have to disagree with that. Yes, ideally we should only believe things for which there is proof, but that is simply not an option for a great many things in our lives and the universe. A lot of the time we have to fall back to estimating how plausible something is based on the knowledge we do have. Even in science it’s common for outcomes to be probabilistic rather than absolute. So I say there is no god because, t…

> There is nothing I have ever seen, or that science has ever collected data on, to indicate that such a thing is plausible. Given that this is an estimate could you estimate what kind of thing you would have to see or what shape of data collected by science that would make you reconsider the plausibility of the existence of a supreme being?

I don't think that's really possible. The issue isn't so much that there isn't proof, it's that proof existing would be counter to everything we know about how the universe works. It wouldn't just mean "oops I'm wrong" it would mean that humanity's perception of reality would have to be fundamentally flawed.

I'm not even opposed to believing that our perception is flawed - clearly we don't know everything and there is much about reality we can't perceive let alone understand. But this would be so far outside of what we do understand that I cannot simply assume that it's true - I would need to see it to believe it.

There are virtually limitless ways such a being could make itself evident to humanity yet the only "evidence" anyone can come up with is either ancient stories or phenomena more plausibly explained by other causes. To me this completely tracks with the implausibility of the existence of god.

Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models

#327

Earlier quoted context omitted.

“ You're objectively correct but judging from your downvotes there seems to be some denial here about that!” I learned upon following Christ and being less liberal that it’s a technique Progressives use. One or more of them ask if there’s any data for the other side. If it doesn’t appear, they’ll say it doesn’t exist. If it does, they try to suppress it with downvotes or deletion. If they succeed, they’ll argue the s…

I think some of it is the training material. People with strong ideologies tend to write more.

True. I’ll modify it with strong motivations to get attention or influence others. The groups that want to be influential the most write the most. From there, it will depend on who spent the most time on the Internet, what was discoverable by search, and bias of recently-popular communities.

That broadens the topics a lot.

Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models

#328
post #55

I'm absolutely amazed at how capable the new 1B model is, considering it's just a 1.3GB download (for the Ollama GGUF version). I tried running a full codebase through it (since it can handle 128,000 tokens) and asking it to summarize the code - it did a surprisingly decent job, incomplete but still unbelievable for a model that tiny: https://gist.github.com/simonw/64c5f5b111fe473999144932bef42... More of my notes he…

I'm new to this game. I played with Gemma 2 9B in an agent-like role before and was pleasantly surprised. I just tried some of the same prompts with Llama 3.2 3B and found it doesn't stick to my instructions very well.

Since I'm a n00b, does this just mean Llama 3.2 3B instruct was "tuned more softly" than Gemma 2 instruct? That is, could one expect to be able to further fine-tune it to more closely follow instructions?

Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models

#329
post #234

Earlier quoted context omitted.

If you are in US, you get 1 billion tokens a DAY with Gemini (Google) completely free of cost. Gemini Flash is fast with upto 4 million token context. Gemini Flash 002 improved in math and logical abilities surpassing Claude and Gpt 4o You can simply use Gemini Flash for Code Completion, git review tool and many more.

Is this sustainable though, or are they just trying really hard to attract users? If I build all of my tooling on it, will they start charging me thousands of dollars next year once the subsidies dry up? With a local model running with open source software, at least I can know that as long as my computer can still compute, the model will still run just as well and just as fast as it did on day 1, and cost the same am…

It's "free cloud picture video storage" rush all over again

Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models

#330

Earlier quoted context omitted.

> There is nothing I have ever seen, or that science has ever collected data on, to indicate that such a thing is plausible. Given that this is an estimate could you estimate what kind of thing you would have to see or what shape of data collected by science that would make you reconsider the plausibility of the existence of a supreme being?

I don't think that's really possible. The issue isn't so much that there isn't proof, it's that proof existing would be counter to everything we know about how the universe works. It wouldn't just mean "oops I'm wrong" it would mean that humanity's perception of reality would have to be fundamentally flawed. I'm not even opposed to believing that our perception is flawed - clearly we don't know everything and there i…

> The issue isn't so much that there isn't proof, it's that proof existing would be counter to everything we know about how the universe works.

I'm not quite sure what you're saying here. It doesn't sound like you're saying that "supreme being" is "black white" (that is, mutually contradictory, meaningless). More like "proof of the existence of the supreme being is impossible". But you also say "I would need to see it to believe it", which suggests that you do think there is a category of proofs that would demonstrate the existence of the supreme being.

Post reply on HN