Live data from Hacker News

Llama 3.2: Revolutionizing edge AI and vision with open, customizable models

ai.meta.com

181–190 of 343 posts

Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models

#181

Earlier quoted context omitted.

bias is some normative lens that some people came up with, but it is purely subjective and is a social construct, that has roots in the area of social justice and has nothing to do with the LLM. the proof is that all critics of AI/LLM have never ever produced a single "unbiased" model. If unbiased model does not exist (at least I never seen an AI/LLM sceptics community produce one), then the concept of bias is useles…

If you forget about the social justice stuff for a minute, there are many other types of bias relevant for an LLM. One example is US-centric bias. If I ask the LLM a question where the answer is one thing in the US and another thing in Germany, you can't really de-bias the model. But ideally you can have it request more details in order to give a good answer.

Yes, but that bias has been present in everything related to computers for decades.

As someone from outside the US, it is quite common to face annoyances like address fields expecting addresses in US format, systems misbehaving and sometimes failing silently if you have two surnames, or accented characters in your personal data, etc. Years go by, tech gets better, but these issues don't go away, they just reappear in different places.

It's funny how some people seem to have discovered this kind of bias and started getting angry with LLMs, which are actually quite OK in this respect.

Not saying that it isn't an issue that should be addressed, just that some people are using it as an excuse to get indignant at AI and it doesn't make much sense. Just like the people who get indignant at AI because ChatGPT collects your input and uses it for training - what do they think social networks have been doing with their input in the last 20 years?

Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models

#182

Earlier quoted context omitted.

Yes. It can converse perfectly normal in German. However when quizzed about German idioms it hallucinates them (in fluent German). Though that's the kind of stuff even larger models often have trouble with. For example if you ask GPT 4 about jokes in German it will give you jokes that depend on word play that only works when translated to English. In normal conversation Llama seems to speak fluent German For Ancient…

Thank you very much for taking your time. Your findings are Amazing! I have used ChatGPT to proofread compositions in German and French lately, but it would have never occurred to me that I should have tested ability to understand idioms, which are the cherry on the cake. I’ll have it a go As for Ancient Greek or Latin, ChatGPT has provided consistent translations and great explanations but its compositions had error…

I have tried to use Llama3-7b and 70b, for Ancient Greek and it is very bad. I will test Llama 3.2, but GPT is great at that. You might want to generate 2 or 3 GPT translations of Ancient Greek and select the best sentences from each one. Alongside with some human corrections, and it is almost unbeatable by any human alone.

Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models

#183

Earlier quoted context omitted.

bias is some normative lens that some people came up with, but it is purely subjective and is a social construct, that has roots in the area of social justice and has nothing to do with the LLM. the proof is that all critics of AI/LLM have never ever produced a single "unbiased" model. If unbiased model does not exist (at least I never seen an AI/LLM sceptics community produce one), then the concept of bias is useles…

If you forget about the social justice stuff for a minute, there are many other types of bias relevant for an LLM. One example is US-centric bias. If I ask the LLM a question where the answer is one thing in the US and another thing in Germany, you can't really de-bias the model. But ideally you can have it request more details in order to give a good answer.

What you call bias, I call simply a representation of a training corpus. There is no broad agreement on how to quantify a bias of the model, other than try one-shot prompts like your "who is the most hated Austrian painter?".

If there was no Germany-specific data in the training corpus - it is not fair to expect LLM to know anything about Germany.

You can check a foundation model from Chinese LLM researchers, and you will most likely see Sino-centric bias just because of the training corpus + synthetic data generation was focused on their native/working language, and their goal was to create foundation model for their language.

I challenge any LLM sceptics - instead of just lazily poking holes in models - create a supposedly better model that reduces bias and lets evaluate your model with specific metrics

Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models

#184
post #70

Still no 14/30b parameter models since llama 2. Seriously killing real usability for power users/diy. The 7/8B models are great for poc and moving to edge for minor use cases … but there’s a big and empty gap till 70b that most people can’t run. The tin foil hat in me is saying this is the compromise the powers that be have agreed too. Basically being “open” but practically gimped for average joe techie. Basically ar…

4090 has 24G So we really need ~40B or G model (two cards) or like a ~20B with some room for context window. 5090 has ??G - still unreleased

Qwen2.5 has a 32B release, and quantised at q5_k_m it *just about" completely fills a 4090.

It's a good model, too.

Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models

#185

Earlier quoted context omitted.

If you forget about the social justice stuff for a minute, there are many other types of bias relevant for an LLM. One example is US-centric bias. If I ask the LLM a question where the answer is one thing in the US and another thing in Germany, you can't really de-bias the model. But ideally you can have it request more details in order to give a good answer.

Yes, but that bias has been present in everything related to computers for decades. As someone from outside the US, it is quite common to face annoyances like address fields expecting addresses in US format, systems misbehaving and sometimes failing silently if you have two surnames, or accented characters in your personal data, etc. Years go by, tech gets better, but these issues don't go away, they just reappear in…

agree with you.

all arguments about supposed bias fall flat when you start asking question about ROI of the "debiasing work".

When you calculate $$$ required to de-bias a model, for example to make LLM recognize Syrian phone numbers: in compute and labor, and compare it to the market opportunity than the ROI is simply not there.

There is a good reason why LLMs are English-specific - because it is the largest market with biggest number of highest paying users for such LLM.

If there is no market demand in "de-biased" model that covers the cost of development, then trying to spend $$$ on de-biasing is pure waste of resources

Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models

#186
Tried the 1B model with the "think step by step" prompt.

It gets "which is larger: 9.11 or 9.9?" right if it manages to mention that decimals need to be compared first in its step-by-step thinking. If it skips mentioning decimals, then it says 9.11 is larger.

It gets the strawberry question wrong even after enumerating all the letters correctly, probably because it can't properly count.

Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models

#187
post #6

I'm blown away with just how open the Llama team at Meta is. It is nice to see that they are not only giving access to the models, but they at the same time are open about how they built them. I don't know how the future is going to go in the terms of models, but I sure am grateful that Meta has taken this position, and are pushing more openness.

They are literally training on all the free personal data you provided, so they owe you this much

Given what I see in Facebook comments I'm surprised the AI doesn't just respond with "Amen. Happy Birthday" to every query.

They're clearly majorly scrubbing things somehow

Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models

#188

"The Llama jumped over the ______!" (Fence? River? Wall? Synagogue?) With 1-hot encoding, the answer is "wall", with 100% probability. Oh, you gave plausibility to "fence" too? WRONG! ENJOY MORE PENALTY, SCRUB! I believe this unforgiving dynamic is why model distillation works well. The original teacher model had to learn via the "hot or cold" game on text answers. But when the child instead imitates the teacher's pr…

yeah i mean that is exactly why distillation works. if you just were one hotting it would be the same as training on same dataset

Re: Llama 3.2: Revolutionizing edge AI and vision with open, customizable models

#190
It's hallucinating so badly, it's kinda hilarious

Literally everything about the quote below is wrong. The problem with these "amazing ground-breaking models" is that you never know which knowledge they actually possess.

--- start quote ---

>>> what is the last version of Ash Framework for Elixir you know My training data goes up to December 2023, and according to the Ash Framework releases, the last stable version that I'm familiar with is:

* *Ash 2.14*: Released on April 8, 2021.

However, it's worth noting that Ash has been merged into Elixir itself, and most of its features have been integrated into the Elixir language and standard libraries.

In fact, Ash was one of the first frameworks to be included in Elixir, and it played a significant role in shaping the language. Many of Ash's core concepts, such as `defstruct/1` and `defmodule/1`, were later adopted by Elixir itself.

...

Keep in mind that Ash is no longer actively maintained as a separate framework, but its legacy continues to influence Elixir and its community.

--- end quote ---

Post reply on HN