Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Llama 3.2: Revolutionizing edge AI and vision with open, customizable models

ai.meta.com

Post reply on HN
Print view View original

111–120 of 343 posts

Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models

#111

Post by forgingahead »

simonw wrote: post #55

I'm absolutely amazed at how capable the new 1B model is, considering it's just a 1.3GB download (for the Ollama GGUF version). I tried running a full codebase through it (since it can handle 128,000 tokens) and asking it to summarize the code - it did a surprisingly decent job, incomplete but still unbelievable for a model that tiny: https://gist.github.com/simonw/64c5f5b111fe473999144932bef42... More of my notes he…

What are people using to check token length of code bases? I'd like to point certain app folders to a local LLM, but no idea how that stuff is calculated? Seems like some strategic prompting (eg: this is a rails app, here is the folder structure with file names, and btw here are the actual files to parse) would be more efficient than just giving it the full app folder? No point giving it stuff from /lib and /vendor for the most part I reckon.

Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models

#112

Post by thefourthchime »

opdahl wrote: post #6

I'm blown away with just how open the Llama team at Meta is. It is nice to see that they are not only giving access to the models, but they at the same time are open about how they built them. I don't know how the future is going to go in the terms of models, but I sure am grateful that Meta has taken this position, and are pushing more openness.

They have a hose of ad money and have nothing to lose doing this.

You can’t say that for the other guys.

Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models

#113

Post by dannyobrien »

n8henrie wrote: post #105
dannyobrien wrote: post #100

Earlier quoted context omitted.

llm is Simon's command line front-end to a lot of the llm apis, local and cloud-based. Along with aider-chat, it's my main interface to any LLM work -- it works well with a chat model, one-off queries, and piping text or output into a llm chain. For people who live on the command line, or are just put-off by web interfaces, it's a godsend. About the only thing I need to look further abroad for is when I'm working mul…

I've only used ollama over cli. As per the parent poster -- do you know if there are advantages over ollama for CLI use? Have you used both?

I haven't used Ollama, but from what I've seen, it seems to operate at a different level of abstraction compared to `llm`. I use `llm` to access both remote and local models through its plugin ecosystem[1]. One of the plugins allows you to use Ollama-served local models. This means you can use the same CLI interface with Ollama[2], as well as with OpenAI, Gemini, Anthropic, llamafile, llamacpp, mlc, and others. I select different models for different purposes. Recently, I've switched my default from OpenAI to Anthropic quite seamlessly.

[1] - https://llm.datasette.io/en/stable/plugins/directory.html#pl... [2] - https://github.com/taketwo/llm-ollama

Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models

#114

Post by freedomben »

If anyone else is looking for the bigger models on ollama and wondering where they are, the Ollama blog post answered that for me. The are "coming soon" so they just aren't ready quite yet[1]. I was a little worried when I couldn't find them but sounds like we just need to be patient.

[1]: https://ollama.com/blog/llama3.2

Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models

#115

Post by acedTrex »

_lvbh wrote: post #103
404mm wrote: post #94

Newbie question, what size model would be needed to have a 10x software engineer skills and no knowledge of the human kind (ie, no need to know how to make a pizza or sequence your DNA). Is there such a model?

10x relative to what? I’ve seen bad developers use AI to 10x their productivity but they still couldn’t come anywhere close to a good developer without AI (granted, this was at a hackathon on pretty advanced optimization research. Maybe there’s more impact on lower skilled tasks)

A bad dev using AI is now 10 times more productive at writing bad code

Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models

#116

Post by talldayo »

thefourthchime wrote: post #112
opdahl wrote: post #6

I'm blown away with just how open the Llama team at Meta is. It is nice to see that they are not only giving access to the models, but they at the same time are open about how they built them. I don't know how the future is going to go in the terms of models, but I sure am grateful that Meta has taken this position, and are pushing more openness.

They have a hose of ad money and have nothing to lose doing this. You can’t say that for the other guys.

I can absolutely say that about Google and Apple.

Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models

#117

Post by xena »

freedomben wrote: post #114

If anyone else is looking for the bigger models on ollama and wondering where they are, the Ollama blog post answered that for me. The are "coming soon" so they just aren't ready quite yet[1]. I was a little worried when I couldn't find them but sounds like we just need to be patient. [1]: https://ollama.com/blog/llama3.2

As a rule of thumb with AI stuff: it either works instantly, or wait a day or two.

Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models

#118

Post by jerieljan »

jackbravo wrote: post #99
simonw wrote: post #55

I'm absolutely amazed at how capable the new 1B model is, considering it's just a 1.3GB download (for the Ollama GGUF version). I tried running a full codebase through it (since it can handle 128,000 tokens) and asking it to summarize the code - it did a surprisingly decent job, incomplete but still unbelievable for a model that tiny: https://gist.github.com/simonw/64c5f5b111fe473999144932bef42... More of my notes he…

I saw that you mention https://github.com/simonw/llm/ . Hadn't seen this before. What is its purpose? And why not use ollama instead?

It looks like a multi-purpose utility in the terminal for bridging together the terminal, your scripts or programs to both local and remote LLM providers.

And it looks very handy! I'll use this myself because I do want to invoke OpenAI and other cloud providers just like I do in ollama and piping things around and this accomplishes that, and more.

https://llm.datasette.io/en/stable/

I guess you can also accomplish similar results if you're just looking for `/chat/completions` and such if you configured something like LiteLLM and connecting that to ollama and any other service.

Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models

#119

Post by GaggiX »

f38zf5vdt wrote: post #106
GaggiX wrote: post #61

Earlier quoted context omitted.

Llama 3.2 vision models don't seem that great if they have to compare them to Claude 3 Haiku or GPT4o-mini. For an open alternative I would use Qwen-2-72B model, it's smaller than the 90B and seems to perform quite better. Also Qwen2-VL-7B as an alternative to Llama-3.2-11B, smaller, better in visual benchmarks and also Apache 2.0. Molmo models: https://huggingface.co/collections/allenai/molmo-66f379e6fe3... , also s…

1. Ignore the benchmarks. I've been A/Bing 11B today with Molmo 72B [1], which itself has an ELO neck-and-neck with GPT4o, and it's even. Because everyone in open source tends to train on validation benchmarks, you really can not trust them. 2. The method of tokenization/adapter is novel and uses many fewer tokens than all comparable CLIP/SigLIP-adapter models, making it _much_ faster. Attention is O(n^2) on memory/c…

How about its performance compare to Qwen-2-72B tho?
Post reply on HN