Llama 3.2: Revolutionizing edge AI and vision with open, customizable models

Post by snovv_crash » Thu, Sep 26, 2024, 10:27 AM UTC

adtac wrote: post #101

a_wild_dandan wrote: post #48

"The Llama jumped over the ______!" (Fence? River? Wall? Synagogue?) With 1-hot encoding, the answer is "wall", with 100% probability. Oh, you gave plausibility to "fence" too? WRONG! ENJOY MORE PENALTY, SCRUB! I believe this unforgiving dynamic is why model distillation works well. The original teacher model had to learn via the "hot or cold" game on text answers. But when the child instead imitates the teacher's pr…

>WRONG! ENJOY MORE PENALTY, SCRUB! Is that true tho? During training, the model predicts {"wall": 0.65, "fence": 0.25, "river": 0.03}. Then backprop modifies the weights such that it produces {"wall": 0.67, "fence": 0.24, "river": 0.02} next time. But it does that with a much richer feedback than WRONG! because we're also telling the model how much more likely "fence" is than "wall" in an indirect way. It's likely mo…

Post by snovv_crash » Thu, Sep 26, 2024, 10:40 AM UTC

zackangelo wrote: post #40

ushakov wrote: post #15

Earlier quoted context omitted.

yes, but only the text-only models! https://www.llama.com/docs/model-cards-and-prompt-formats/ll...

This is incorrect: > With text-only inputs, the Llama 3.2 Vision Models can do tool-calling exactly like their Llama 3.1 Text Model counterparts. You can use either the system or user prompts to provide the function definitions. > Currently the vision models don’t support tool-calling with text+image inputs. They support it, but not when an image is submitted in the prompt. I'd be curious to see what the model does.…

Post by jona-f » Thu, Sep 26, 2024, 10:47 AM UTC

Eisenstein wrote: post #130

dannyobrien wrote: post #102

Earlier quoted context omitted.

What interface do you use for a locally-run Qwen2-VL-7B? Inspired by Simon Willison's research[1], I have tried it out on Hugging Face[2]. Its handwriting recognition seems fantastic, but I haven't figured out how to run it locally yet. [1] https://simonwillison.net/2024/Sep/4/qwen2-vl/ [2] https://huggingface.co/spaces/GanymedeNil/Qwen2-VL-7B

MiniCPM-V 2.6 is based on Qwen 2 and is also great at handwriting. It works locally with KoboldCPP. Here are the results I got with a test I just did. Image: * https://imgur.com/wg0kdQK Output: * https://pastebin.com/RKvYQasi OCR script used: * https://github.com/jabberjabberjabber/LLMOCR/blob/main/llmoc... Model weights: MiniCPM-V-2_6-Q6_K_L.gguf, mmproj-MiniCPM-V-2_6-f16.gguf Inference: * https://github.com/LostRui…

Post by nycdatasci » Thu, Sep 26, 2024, 10:55 AM UTC

faangguyindia wrote: post #203

simonw wrote: post #55

I'm absolutely amazed at how capable the new 1B model is, considering it's just a 1.3GB download (for the Ollama GGUF version). I tried running a full codebase through it (since it can handle 128,000 tokens) and asking it to summarize the code - it did a surprisingly decent job, incomplete but still unbelievable for a model that tiny: https://gist.github.com/simonw/64c5f5b111fe473999144932bef42... More of my notes he…

If you are in US, you get 1 billion tokens a DAY with Gemini (Google) completely free of cost. Gemini Flash is fast with upto 4 million token context. Gemini Flash 002 improved in math and logical abilities surpassing Claude and Gpt 4o You can simply use Gemini Flash for Code Completion, git review tool and many more.

Post by boppo1 » Thu, Sep 26, 2024, 11:14 AM UTC

warkdarrior wrote: post #65

nickpsecurity wrote: post #10

Earlier quoted context omitted.

Do they tell you what training data they use for alignment? As in, what biases they intentionally put in the system they’re widely deploying?

Do you have some concrete example of biases in their models? Or are you just fishing for something to complain about?