Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Llama 3.2: Revolutionizing edge AI and vision with open, customizable models

ai.meta.com

Post reply on HN
Print view View original

141–150 of 343 posts

Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models

#141

Post by SOLAR_FIELDS »

dannyobrien wrote: post #100
jackbravo wrote: post #99

Earlier quoted context omitted.

I saw that you mention https://github.com/simonw/llm/ . Hadn't seen this before. What is its purpose? And why not use ollama instead?

llm is Simon's command line front-end to a lot of the llm apis, local and cloud-based. Along with aider-chat, it's my main interface to any LLM work -- it works well with a chat model, one-off queries, and piping text or output into a llm chain. For people who live on the command line, or are just put-off by web interfaces, it's a godsend. About the only thing I need to look further abroad for is when I'm working mul…

I use a fair amount of aider - what does Simon's solution offer that aider doesn't? I am usually using a mix of aider and the ChatGPT window. I use ChatGPT for one off queries that aren't super context heavy for my codebase, since pricing can still add up for the API and a lot of the times the questions that I ask don't really need deep context about what I'm doing in the terminal. But when I'm in flow state and I need deep integration with the files I'm changing I switch over to aider with Sonnet - my subjective experience is that Anthropic's models are significantly better for that use case. Curious if Simon's solution is more geared toward the first use case or the second.

Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models

#142

Post by nextworddev »

opdahl wrote: post #6

I'm blown away with just how open the Llama team at Meta is. It is nice to see that they are not only giving access to the models, but they at the same time are open about how they built them. I don't know how the future is going to go in the terms of models, but I sure am grateful that Meta has taken this position, and are pushing more openness.

They are literally training on all the free personal data you provided, so they owe you this much

Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models

#143

Post by sunaookami »

mistrial9 wrote: post #125
nickpsecurity wrote: post #122

Earlier quoted context omitted.

Google’s and OpenAI often answered far-left, Progressive, and atheist. Google’s was censoring white people at one point. Facebook seems to espouse similar values. They’ve funded work to increase those values. Many mention topics relevant to these things in their papers in the bias or alignment sections. These political systems don’t represent the majority of the world. They might not even represent half the U.S.. Peo…

this provocative parent-post may or may not be accurate, but what is missing IMHO is any characterization of the question asked or other context of use.. lacking that basic part to the inquiry, this statement alone is clearly amateurish, zealous and as said, provocative. Fighting in words is too easy! like falling off a log, as they say.. in politics it is almost unavoidable. Please, not start fires. All that said ye…

>This forum is worth better questions

That's not for you to decide if some question is "worth". At least for OpenAI and Anthropic it is a fact that these models are pre-censored by the US government: https://www.cnbc.com/2024/08/29/openai-and-anthropic-agree-t...

Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models

#144

Post by benreesman »

f38zf5vdt wrote: post #106
GaggiX wrote: post #61

Earlier quoted context omitted.

Llama 3.2 vision models don't seem that great if they have to compare them to Claude 3 Haiku or GPT4o-mini. For an open alternative I would use Qwen-2-72B model, it's smaller than the 90B and seems to perform quite better. Also Qwen2-VL-7B as an alternative to Llama-3.2-11B, smaller, better in visual benchmarks and also Apache 2.0. Molmo models: https://huggingface.co/collections/allenai/molmo-66f379e6fe3... , also s…

1. Ignore the benchmarks. I've been A/Bing 11B today with Molmo 72B [1], which itself has an ELO neck-and-neck with GPT4o, and it's even. Because everyone in open source tends to train on validation benchmarks, you really can not trust them. 2. The method of tokenization/adapter is novel and uses many fewer tokens than all comparable CLIP/SigLIP-adapter models, making it _much_ faster. Attention is O(n^2) on memory/c…

It’s not just open source that trains on the validation set. The big labs have already forgotten more about gaming MMLU down to the decimal than the open source community ever knew. Every once in a while they get sloppy and Claude does a faux pas with a BIGBENCH canary string or some other embarrassing little admission of dishonesty like that.

A big lab gets exactly the score on any public eval that they want to. They have their own holdouts for actual ML work, and they’re some of the most closely guarded IP artifacts, far more valuable than a snapshot of weights.

Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models

#145

Post by v3ss0n »

simonw wrote: post #139
lowyek wrote: post #134

Earlier quoted context omitted.

Hi simon, is there a way to run the vision model easily on my mac locally?

Not that I’ve seen so far, but Ollama are pending a solution for that “soon”.

I doubt ollama team can do much about it. Ollama are just wrapper on top of heavy lifter

Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models

#146

Post by alexcpn »

In KungfuPanda there is this line that the Panda says "I love KungFuuuuuuuu", well I normally don't tell like this, but when I saw this and (starting to use this), I feel like yelling"I like Metaaaaa or is it LLAMMMAAA or is it Open source.. or is it this cool ecosystem which gives such value for free...

Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models

#147

Post by slt2021 »

ericjmorey wrote: post #93
warkdarrior wrote: post #65

Earlier quoted context omitted.

Do you have some concrete example of biases in their models? Or are you just fishing for something to complain about?

Even without intentionally biasing the model, without knowing the biases that exist in the training data, they're just biased black boxes that come with the overhead of figuring out how it's biased. All data is biased, there's no avoiding that fact.

bias is some normative lens that some people came up with, but it is purely subjective and is a social construct, that has roots in the area of social justice and has nothing to do with the LLM.

the proof is that all critics of AI/LLM have never ever produced a single "unbiased" model. If unbiased model does not exist (at least I never seen an AI/LLM sceptics community produce one), then the concept of bias is useless.

Just a fluffy word that does not mean anything

Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models

#148

Post by alanzhuly »

Llama3.2 3B feels a lot better than other models with same size (e.g. Gemma2, Phi3.5-mini models).

For anyone looking for a simple way to test Llama3.2 3B locally with UI, Install nexa-sdk(https://github.com/NexaAI/nexa-sdk) and type in terminal:

nexa run llama3.2 --streamlit

Disclaimer: I am from Nexa AI and nexa-sdk is an open-sourced. We'd love your feedback.

Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models

#150

Post by imjonse »

opdahl wrote: post #6

I'm blown away with just how open the Llama team at Meta is. It is nice to see that they are not only giving access to the models, but they at the same time are open about how they built them. I don't know how the future is going to go in the terms of models, but I sure am grateful that Meta has taken this position, and are pushing more openness.

Training data is crucial for performance and they do not (cannot) share that.
Post reply on HN