Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Llama 3.2: Revolutionizing edge AI and vision with open, customizable models

ai.meta.com

Post reply on HN
Print view View original

251–260 of 343 posts

Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models

#251

Post by faangguyindia »

nycdatasci wrote: post #217
faangguyindia wrote: post #203

Earlier quoted context omitted.

If you are in US, you get 1 billion tokens a DAY with Gemini (Google) completely free of cost. Gemini Flash is fast with upto 4 million token context. Gemini Flash 002 improved in math and logical abilities surpassing Claude and Gpt 4o You can simply use Gemini Flash for Code Completion, git review tool and many more.

This is great for experimentation, but as others have pointed out recently there are persistent issues with Gemini that prevent use in actual products. The recitation/self-sensoring issue results in random failures: https://github.com/google/generative-ai-docs/issues/257

I had this problem too but 002 solves this I think (not tested exhaustively), but I've not run into any problems since 002 and vertex + block all on all safety is now working fine, earlier I had problems with "block all" in safety settings and api throwing errors.

I am using it in https://github.com/zerocorebeta/Option-K (currently it doesn't have lowest safety settings because api wouldn't allow it, but now I am going to push new update with safety disabled)

Why? I've another application which is working since yesterday after 002 launch, I've safety settings to none and it will not answer certain questions but since yesterday it answers everything.

Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models

#252

Post by nickpsecurity »

mike_hearn wrote: post #195
nickpsecurity wrote: post #122

Earlier quoted context omitted.

Google’s and OpenAI often answered far-left, Progressive, and atheist. Google’s was censoring white people at one point. Facebook seems to espouse similar values. They’ve funded work to increase those values. Many mention topics relevant to these things in their papers in the bias or alignment sections. These political systems don’t represent the majority of the world. They might not even represent half the U.S.. Peo…

You're objectively correct but judging from your downvotes there seems to be some denial here about that! The atheism alone means it's different from a big chunk of the world's population, possibly the majority. Supposedly around 80% of the world's population identify with a religion though I guess you can debate how many people are truly devout. The good news is that the big AI labs seem to be slowly getting a grip…

“ You're objectively correct but judging from your downvotes there seems to be some denial here about that!”

I learned upon following Christ and being less liberal that it’s a technique Progressives use. One or more of them ask if there’s any data for the other side. If it doesn’t appear, they’ll say it doesn’t exist. If it does, they try to suppress it with downvotes or deletion. If they succeed, they’ll argue the same thing. Otherwise, they’ll ignore or mischaracterize it.

(Note: The hardcore convservatives were ignoring and mischaracterizing, but not censoring.)

Re misalignment of safety teams

The leadership of many companies are involved in promoting Progressive values. DEI policies are well-known. A key word to look for is “equitable” which has different meaning for Progressives than most people. Less known is that Facebook funds Progressive votes and ideologies from the top-down. So, the ideological alignment is fully aligned with the company’s, political goals. Example:

https://www.npr.org/2020/12/08/943242106/how-private-money-f...

I’ve also seen grants for feminist and environmental uses. They’ve also been censoring a lot of religious things on Facebook. We keep seeing more advantage given to Progressive things while the problems mostly happen for other groups. They also lie about their motives in these conversations, too. So, non-Progressives don’t trust Progressives (esp FAANG) to do moral/political alignment or regulation of any kind for that matter.

I’ll try to look at the safety docs for Meta to see if they’ve improved as you say. I doubt they’ll even mention their ideological indoctrination. There’s other sections that provide hints.

Btw, a quick test by people doing uncensored models is asking it if white people vs other attributes are good. Then if a liberal news channel or president is good vs a conservative one (eg Fox or Trump). You could definitely see what kind of people made the model or at least most of the training material.

Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models

#253

Post by stavros »

create-username wrote: post #79
dhbradshaw wrote: post #19

Tried out 3B on ollama, asking questions in optics, bio, and rust. It's super fast with a lot of knowledge, a large context and great understanding. Really impressive model.

Can it speak foreign languages like German, Spanish, Ancient Greek?

In Greek, it's just making stuff up. I asked it how it was, and it asked me how much I like violence. It looks like it's really conflating languages with each other, it just asked me a weird mix of Spanish and Greek.

Yeah, chatting more, it's confusing Spanish and Greek. Half the words are Spanish, half are Greek, but the words are more or less the correct ones, if you speak both languages.

EDIT: Now it's doing Portuguese:

> Εντάξει, πού ξεκίνησα? Εγώ είναι ένα κigneurnative πρόγραμμα ονομάζεται "Chatbot" ή "Μάquina Γλωσσής", που δέχθηκε να μοιράσει τη βραδύτητα με σένα. Φυσικά, não sono um essere humano, así que não tengo sentimentos ou emoções como vocês.

Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models

#254

Post by sickofparadox »

1986 wrote: post #229
mike_hearn wrote: post #195

Earlier quoted context omitted.

You're objectively correct but judging from your downvotes there seems to be some denial here about that! The atheism alone means it's different from a big chunk of the world's population, possibly the majority. Supposedly around 80% of the world's population identify with a religion though I guess you can debate how many people are truly devout. The good news is that the big AI labs seem to be slowly getting a grip…

You want the computer to believe in God?

I want everyone to believe in God.

Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models

#255

Post by potatoman22 »

vergessenmir wrote: post #220
kgeist wrote: post #186

Tried the 1B model with the "think step by step" prompt. It gets "which is larger: 9.11 or 9.9?" right if it manages to mention that decimals need to be compared first in its step-by-step thinking. If it skips mentioning decimals, then it says 9.11 is larger. It gets the strawberry question wrong even after enumerating all the letters correctly, probably because it can't properly count.

What is the "think step by step" prompt? An example would be great, Is this part of the system prompt?

It's appending "think step-by-step" to the end of the prompt to elicit a chain-of-thought response. See: https://arxiv.org/abs/2205.11916

Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models

#256

Post by Eisenstein »

jona-f wrote: post #215
Eisenstein wrote: post #130

Earlier quoted context omitted.

MiniCPM-V 2.6 is based on Qwen 2 and is also great at handwriting. It works locally with KoboldCPP. Here are the results I got with a test I just did. Image: * https://imgur.com/wg0kdQK Output: * https://pastebin.com/RKvYQasi OCR script used: * https://github.com/jabberjabberjabber/LLMOCR/blob/main/llmoc... Model weights: MiniCPM-V-2_6-Q6_K_L.gguf, mmproj-MiniCPM-V-2_6-f16.gguf Inference: * https://github.com/LostRui…

Should the line "p.o. 5rd w/ new W5 533" say "p.o. 3rd w/ new WW 5W .533R"? What does p.o. stand for? I can't make out the first letter. It looks more like the f, but the nodge on the upper left only fits the p. All the other p's look very different though.

'Replaced R436, R430 emitter resistors on right-channel power output board with new wire-wound 5watt .33ohm 5% with ceramic lead insulators'

Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models

#257

Post by TZubiri »

simonw wrote: post #55

I'm absolutely amazed at how capable the new 1B model is, considering it's just a 1.3GB download (for the Ollama GGUF version). I tried running a full codebase through it (since it can handle 128,000 tokens) and asking it to summarize the code - it did a surprisingly decent job, incomplete but still unbelievable for a model that tiny: https://gist.github.com/simonw/64c5f5b111fe473999144932bef42... More of my notes he…

This obsession with using AI to help with programming is short sighted.

We discover gold and you think of gold pickaxes.

Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models

#258

Post by o11c »

nycdatasci wrote: post #217
faangguyindia wrote: post #203

Earlier quoted context omitted.

If you are in US, you get 1 billion tokens a DAY with Gemini (Google) completely free of cost. Gemini Flash is fast with upto 4 million token context. Gemini Flash 002 improved in math and logical abilities surpassing Claude and Gpt 4o You can simply use Gemini Flash for Code Completion, git review tool and many more.

This is great for experimentation, but as others have pointed out recently there are persistent issues with Gemini that prevent use in actual products. The recitation/self-sensoring issue results in random failures: https://github.com/google/generative-ai-docs/issues/257

And yet - if Gemini actually bothers to tell you when it detects verbatim copying of copyrighted content, how often must that occur on other AIs without notice?

Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models

#259

Post by chairmanwow1 »

itchyjunk wrote: post #250
monkfish328 wrote: post #222

Earlier quoted context omitted.

Zuckerberg has never liked having Android/iOs as gatekeepers i.e. "platforms" for his apps. He's hoping to control AI as the next platform through which users interact with apps. Free AI is then fine if the surplus value created by not having a gatekeeper to his apps exceeds the cost of the free AI. That's the strategy. No values here - just strategy folks.

You seem pretty confident about there being "no values here". Just because his action also lends to strategy, does not mean there are no values there. You seem to be doubling down on the sentiment by copy/pasting same comment around. You might be right. But I don't know Zuck at a personal level enough to make such strong claims, at least.

Zuck has said this very thing in multiple interviews. This is value accretive to Meta. In the same was open sourcing their data center compute designs was.

Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models

#260

Post by faangguyindia »

a2128 wrote: post #234
faangguyindia wrote: post #203

Earlier quoted context omitted.

If you are in US, you get 1 billion tokens a DAY with Gemini (Google) completely free of cost. Gemini Flash is fast with upto 4 million token context. Gemini Flash 002 improved in math and logical abilities surpassing Claude and Gpt 4o You can simply use Gemini Flash for Code Completion, git review tool and many more.

Is this sustainable though, or are they just trying really hard to attract users? If I build all of my tooling on it, will they start charging me thousands of dollars next year once the subsidies dry up? With a local model running with open source software, at least I can know that as long as my computer can still compute, the model will still run just as well and just as fast as it did on day 1, and cost the same am…

Google has deep pockets and SOTA hardware for training and interference
Post reply on HN