311–320 of 695 posts
Earlier quoted context omitted.
17B puts it beyond the reach of a 4090 ... anybody do 4 bit quant on it yet?
Unless something’s changed you will need the whole model on the HPU anyway, no? So way beyond a 4090 regardless.
Also I see the 4 bit quants put it at a h100 which is fine ... I've got those at work. Maybe there will be distilled for running at home
Earlier quoted context omitted.
This was an idea that sounded somewhat silly until it was shown it worked. The idea is that you encourage through training a bunch of “experts” to diversify and “get good” at different things. These experts are say 1/10 to 1/100 of your model size if it were a dense model. So you pack them all up into one model, and you add a layer or a few layers that have the job of picking which small expert model is best for your…
The only thing about this which may be unintuitive from the name is an "Expert" is not something like a sub-llm that's good at math and gets called when you ask a math question. Models like this have layers of networks they run tokens through and each layer is composed of 256 sub-networks, any of which can be selected (or multiple selected and merged in some way) for each layer independently. So the net result is the…
https://www.llama.com/ https://www.llama.com/docs/model-cards-and-prompt-formats/ll... Very exciting. Benchmarks look good, and most importantly it looks like they did a lot of work improving vision performance (based on benchmarks). The new suggested system prompt makes it seem like the model is less censored, which would be great. The phrasing of the system prompt is ... a little disconcerting in context (Meta's ko…
It seems to be less censored than Llama 3, and can describe NSFW images and interact with them. It did refuse me once, but complied after reminding it of its system prompt. Accuracy of visual NSFW content is not particularly good; much worse than GPT 4o.
More "sensitive" requests, like asking it to guess the political affiliation of a person from an image, required a _lot_ of coaxing in the system prompt. Otherwise it tends to refuse. Even with their suggested prompt that seemingly would have allowed that.
More extreme prompts, like asking it to write derogatory things about pictures of real people, took some coaxing as well but was quite straight-forward.
So yes, I'd say this iteration is less censored. Vision is better, but OpenAI and Qwen still lead the pack.
Earlier quoted context omitted.
17B puts it beyond the reach of a 4090 ... anybody do 4 bit quant on it yet?
Unless something’s changed you will need the whole model on the HPU anyway, no? So way beyond a 4090 regardless.
see ktransformers: https://www.reddit.com/r/LocalLLaMA/comments/1jpi0n9/ktransf...
Earlier quoted context omitted.
This was an idea that sounded somewhat silly until it was shown it worked. The idea is that you encourage through training a bunch of “experts” to diversify and “get good” at different things. These experts are say 1/10 to 1/100 of your model size if it were a dense model. So you pack them all up into one model, and you add a layer or a few layers that have the job of picking which small expert model is best for your…
The only thing about this which may be unintuitive from the name is an "Expert" is not something like a sub-llm that's good at math and gets called when you ask a math question. Models like this have layers of networks they run tokens through and each layer is composed of 256 sub-networks, any of which can be selected (or multiple selected and merged in some way) for each layer independently. So the net result is the…
I think where MoE is misleading is that the experts aren't what we would call "experts" in the normal world but rather they are experts for a specific token. that concept feels difficult to grasp.
"It’s well-known that all leading LLMs have had issues with bias—specifically, they historically have leaned left when it comes to debated political and social topics. This is due to the types of training data available on the internet." Perhaps. Or, maybe, "leaning left" by the standards of Zuck et al. is more in alignment with the global population. It's a simpler explanation.
I find it impossible to discuss bias without a shared understanding of what it actually means to be unbiased - or at least, a shared understanding of what the process of reaching an unbiased position looks like. 40% of Americans believe that God created the earth in the last 10,000 years. If I ask an LLM how old the Earth is, and it replies ~4.5 billion years old, is it biased?
It is of course a radical left lunatic LLM.
10M Context Window with such a cheap performance WHILE having one of the top LMArena scores is really impressive. The choice to have 128 experts is also unseen as far as I know, right? But seems to have worked pretty good as it seems.
Like, if you consulted 128 actual experts, you'd get something way better than any LLM output.
Earlier quoted context omitted.
Not that I agree with all the linked points but it is weird to me that LeCun consistently states LLMs are not the right path yet LLMs are still the main flagship model they are shipping. Although maybe he's using an odd definition for what counts as a LLM. https://www.threads.net/@yannlecun/post/DD0ac1_v7Ij?hl=en
> LeCun consistently states LLMs are not the right path yet LLMs are still the main flagship model they are shipping. I really don't see what's controversial about this. If that's to mean that LLMs are inherently flawed/limited and just represent a local maxima in the overall journey towards developing better AI techniques, I thought that was pretty universal understanding by now.
Llama 4 Scout: 210GB
FYI.
General overview below, as the pages don't seem to be working well Llama 4 Models: - Both Llama 4 Scout and Llama 4 Maverick use a Mixture-of-Experts (MoE) design with 17B active parameters each. - They are natively multimodal: text + image input, text-only output. - Key achievements include industry-leading context lengths, strong coding/reasoning performance, and improved multilingual capabilities. - Knowledge cuto…