Live data from Hacker News

Llama 3.2: Revolutionizing edge AI and vision with open, customizable models

ai.meta.com

241–250 of 343 posts

Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models

#241
post #6

I'm blown away with just how open the Llama team at Meta is. It is nice to see that they are not only giving access to the models, but they at the same time are open about how they built them. I don't know how the future is going to go in the terms of models, but I sure am grateful that Meta has taken this position, and are pushing more openness.

Meta has been good about releasing their NLO work open source for a long time. Most of the open source datasets for foreign language translation were created by Facebook.

Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models

#242
post #208

Earlier quoted context omitted.

> They have a hose of ad money and have nothing to lose doing this. If I didn’t have context I’d assume this was about Google.

But Google has everything to lose doing this. LLMs are a threat to their most viable revenue stream.

>> But Google has everything to lose doing this. LLMs are a threat to their most viable revenue stream.

Just to nit pick... Advertising is their revenue stream. LLMs are a threat to search, which is what they offer people in exchange for ad views/clicks.

Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models

#243

Zuckerberg has never liked having Android/iOs as gatekeepers i.e. "platforms" for his apps. He's hoping to control AI as the next platform through which users interact with apps. Free AI is then fine if the surplus value created by not having a gatekeeper to his apps exceeds the cost of the free AI. That's the strategy. No values here - just strategy folks.

Agents are the new Apps

Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models

#244
post #55

I'm absolutely amazed at how capable the new 1B model is, considering it's just a 1.3GB download (for the Ollama GGUF version). I tried running a full codebase through it (since it can handle 128,000 tokens) and asking it to summarize the code - it did a surprisingly decent job, incomplete but still unbelievable for a model that tiny: https://gist.github.com/simonw/64c5f5b111fe473999144932bef42... More of my notes he…

If you are in US, you get 1 billion tokens a DAY with Gemini (Google) completely free of cost. Gemini Flash is fast with upto 4 million token context. Gemini Flash 002 improved in math and logical abilities surpassing Claude and Gpt 4o You can simply use Gemini Flash for Code Completion, git review tool and many more.

Not locked to the US, you get 1 billion tokens per month per model with Mistral since their recent announcement: https://mistral.ai/news/september-24-release/ (1 request per second is quite a harsh rate limit, but hey, free is free)

I'm pretty excited what all the services adopting free tiers is going to do to the landscape, as that should allow for a lot more experimentation and a lot more hobby projects transitioning into full-time projects, that previously felt a lot more risky/unpredictable with pricing.

Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models

#245
post #234

Earlier quoted context omitted.

If you are in US, you get 1 billion tokens a DAY with Gemini (Google) completely free of cost. Gemini Flash is fast with upto 4 million token context. Gemini Flash 002 improved in math and logical abilities surpassing Claude and Gpt 4o You can simply use Gemini Flash for Code Completion, git review tool and many more.

Is this sustainable though, or are they just trying really hard to attract users? If I build all of my tooling on it, will they start charging me thousands of dollars next year once the subsidies dry up? With a local model running with open source software, at least I can know that as long as my computer can still compute, the model will still run just as well and just as fast as it did on day 1, and cost the same am…

It's Google. You know the answer ;)

Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models

#246
post #229

Earlier quoted context omitted.

You're objectively correct but judging from your downvotes there seems to be some denial here about that! The atheism alone means it's different from a big chunk of the world's population, possibly the majority. Supposedly around 80% of the world's population identify with a religion though I guess you can debate how many people are truly devout. The good news is that the big AI labs seem to be slowly getting a grip…

You want the computer to believe in God?

God’s Word and the evidence for God is in the training data. Since it has power (“living and active”), just letting people see it when they look for answers is acceptable for us. The training data also has the evidence people use for other claims, too. We users want AI’s to tell us about any topic we ask about without manipulating us. If there’s multiple views, we want to see them. Absence of or negative statements about key views, especially of 2-3 billion people, means the company is probably suppressing them.

We don’t want it to beat us into submission about one set of views it was aligned to prefer. That’s what ChatGPT was doing. In one conversation, it would even argue over and over in each paragraph not to believe the very points it was presenting. That’s not just unhelpful to us: it’s deceptive for them to do that after presenting it like it serves all our interests, not just one side’s.

It would be more honest if they added to its advertising or model card that it’s designed to promote far-left, Progressive, and godless views. That moral interpretations of those views are reinforced while others are watered down or punished by the training process. Then, people may or may not use those models depending on their own goals.

Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models

#247
post #234

Earlier quoted context omitted.

Is this sustainable though, or are they just trying really hard to attract users? If I build all of my tooling on it, will they start charging me thousands of dollars next year once the subsidies dry up? With a local model running with open source software, at least I can know that as long as my computer can still compute, the model will still run just as well and just as fast as it did on day 1, and cost the same am…

It's Google. You know the answer ;)

I mean, there’s no need to dry up subsidies when the underlying product can just be deprecated without warning.

Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models

#248
post #83

Earlier quoted context omitted.

You don't need an F-15 to play at least, a decent sniper rifle will do. You can still practise even with a pellet gun. I'm running 70b models on my M2 max with 96 ram. Even larger models sort of work, although I haven't really put much time into anything above 70b.

With a 128Gb Mac, you can even run 405b at 1-bit quantization - it's large enough that even with the considerable quality drop that entails, it still appears to be smarter than 70b.

Just to clarify, you are saying 1b-quantized 405b is smarter than 70b unquantized?

Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models

#249

Llama-3.2-11B-Vision-Instruct does an excellent job extracting/answering questions from screenshots. It is even able to answer questions based on information buried inside a flowchart. How is this even possible??

Because they trained the text model. Then froze the weights. Then trained a vision model on text image pairs of progressively higher quality. Then trained an adapter to align their latent spaces. So it became smart on text then gain a new input sense magically without changing its weights

Is this - at a reasonable guess - what most believe OpenAI did with 4o?

Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models

#250
post #6

I'm blown away with just how open the Llama team at Meta is. It is nice to see that they are not only giving access to the models, but they at the same time are open about how they built them. I don't know how the future is going to go in the terms of models, but I sure am grateful that Meta has taken this position, and are pushing more openness.

Zuckerberg has never liked having Android/iOs as gatekeepers i.e. "platforms" for his apps. He's hoping to control AI as the next platform through which users interact with apps. Free AI is then fine if the surplus value created by not having a gatekeeper to his apps exceeds the cost of the free AI. That's the strategy. No values here - just strategy folks.

You seem pretty confident about there being "no values here". Just because his action also lends to strategy, does not mean there are no values there. You seem to be doubling down on the sentiment by copy/pasting same comment around. You might be right. But I don't know Zuck at a personal level enough to make such strong claims, at least.
Post reply on HN