Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Llama 3.2: Revolutionizing edge AI and vision with open, customizable models

ai.meta.com

Post reply on HN
Print view View original

301–310 of 343 posts

Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models

#301

Post by flakiness »

jackbravo wrote: post #99
simonw wrote: post #55

I'm absolutely amazed at how capable the new 1B model is, considering it's just a 1.3GB download (for the Ollama GGUF version). I tried running a full codebase through it (since it can handle 128,000 tokens) and asking it to summarize the code - it did a surprisingly decent job, incomplete but still unbelievable for a model that tiny: https://gist.github.com/simonw/64c5f5b111fe473999144932bef42... More of my notes he…

I saw that you mention https://github.com/simonw/llm/ . Hadn't seen this before. What is its purpose? And why not use ollama instead?

There is a recent podcast episode with the tool's author https://newsletter.pragmaticengineer.com/p/ai-tools-for-soft...

It's worth listening to learn abouut the context on how that tool is used.

Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models

#302

Post by HDThoreaun »

monkfish328 wrote: post #222
opdahl wrote: post #6

I'm blown away with just how open the Llama team at Meta is. It is nice to see that they are not only giving access to the models, but they at the same time are open about how they built them. I don't know how the future is going to go in the terms of models, but I sure am grateful that Meta has taken this position, and are pushing more openness.

Zuckerberg has never liked having Android/iOs as gatekeepers i.e. "platforms" for his apps. He's hoping to control AI as the next platform through which users interact with apps. Free AI is then fine if the surplus value created by not having a gatekeeper to his apps exceeds the cost of the free AI. That's the strategy. No values here - just strategy folks.

You can recognize this and still be grateful that Mark's incentives align with my own in a way that has made llama free and open sourceish

Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models

#303

Post by xyc »

forgingahead wrote: post #111
simonw wrote: post #55

I'm absolutely amazed at how capable the new 1B model is, considering it's just a 1.3GB download (for the Ollama GGUF version). I tried running a full codebase through it (since it can handle 128,000 tokens) and asking it to summarize the code - it did a surprisingly decent job, incomplete but still unbelievable for a model that tiny: https://gist.github.com/simonw/64c5f5b111fe473999144932bef42... More of my notes he…

What are people using to check token length of code bases? I'd like to point certain app folders to a local LLM, but no idea how that stuff is calculated? Seems like some strategic prompting (eg: this is a rails app, here is the folder structure with file names, and btw here are the actual files to parse) would be more efficient than just giving it the full app folder? No point giving it stuff from /lib and /vendor f…

You can use llama.cpp server's tokenize endpoint to tokenize and count the tokens: https://github.com/ggerganov/llama.cpp/blob/master/examples/...

Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models

#305

Post by altruios »

kgeist wrote: post #186

Tried the 1B model with the "think step by step" prompt. It gets "which is larger: 9.11 or 9.9?" right if it manages to mention that decimals need to be compared first in its step-by-step thinking. If it skips mentioning decimals, then it says 9.11 is larger. It gets the strawberry question wrong even after enumerating all the letters correctly, probably because it can't properly count.

My understanding is the way the tokenization works prevents the LMM from being able to count occurrences of words or individual characters.

Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models

#307

Post by kristopolous »

euroderf wrote: post #263
kristopolous wrote: post #187

Earlier quoted context omitted.

Given what I see in Facebook comments I'm surprised the AI doesn't just respond with "Amen. Happy Birthday" to every query. They're clearly majorly scrubbing things somehow

In a few years (or months?) Faceborg will offer a new service "EverYou" trained on your entire Faceborg corpus. It will speak like you to others (whomever you permit) and it will like what you like (acting as a web gopher for you) and it will be able stay up late talking to tipsy you about life, the universe, and everything, and it will be... "long-term affordable".

facebook knows me so poorly though. I just look at the suggested posts. It's stuff like celebrity gossip, sports, and troop worshiping ai images. I've been on the platform 20 years and I've never posted about any of this stuff. I don't know who or what they're talking about. It's just a never-ending stream of stuff I have no interest in.

Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models

#309

Post by threatripper »

arnaudsm wrote: post #87

Is there an up-to-date leaderboard with multiple LLM benchmarks? Livebench and Lmsys are weeks behind and sometimes refuse to add some major models. And press releases like this cherry pick their benchmarks and ignore better models like qwen2.5. If it doesn't exist I'm willing to create it

https://artificialanalysis.ai/leaderboards/models

"LLM Leaderboard - Comparison of GPT-4o, Llama 3, Mistral, Gemini and over 30 models

Comparison and ranking the performance of over 30 AI models (LLMs) across key metrics including quality, price, performance and speed (output speed - tokens per second & latency - TTFT), context window & others. For more details including relating to our methodology, see our FAQs."

Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models

#310

Post by halJordan »

chairmanwow1 wrote: post #259
itchyjunk wrote: post #250

Earlier quoted context omitted.

You seem pretty confident about there being "no values here". Just because his action also lends to strategy, does not mean there are no values there. You seem to be doubling down on the sentiment by copy/pasting same comment around. You might be right. But I don't know Zuck at a personal level enough to make such strong claims, at least.

Zuck has said this very thing in multiple interviews. This is value accretive to Meta. In the same was open sourcing their data center compute designs was.

The world doesn't exist in black and white. When you force the shades of grey to be binary your choosing force your conclusion onto the data rather take your conclusions from the data.

Thats not to say there isnt a strategy or that it's all values. Its to say that youre denying Zuck any chance at values because you enjoy hating on him. Bc Zuck has also said in multiple interviews that his values do include open source and given two facts with the same level of sourcing you deny the one fact that doesn't let you be mean.

Post reply on HN