This is probably a better link. https://www.llama.com/docs/model-cards-and-prompt-formats/ll...
That link doesn't work
Works for me
The Llama 4 herd
41–50 of 695 posts
Re: The Llama 4 herd
#42The blog post is quite informative: https://ai.meta.com/blog/llama-4-multimodal-intelligence/
Re: The Llama 4 herd
#43What an electrifying time to be alive! The last era that felt even remotely this dynamic was during the explosive rise of JavaScript frameworks—when it seemed like a new one dropped every quarter. Back then, though, the vibe was more like, “Ugh, another framework to learn?” Fast forward to now, and innovation is sprinting forward again—but this time, it feels like a thrilling ride we can’t wait to be part of.
Re: The Llama 4 herd
#44As expected, Meta doesn't disappoint and accelerates the race to zero. Meta is undervalued.
How does Meta make money from Llama?
Re: The Llama 4 herd
#45The (smaller) Scout model is really attractive for Apple Silicon. It is 109B big but split up into 16 experts. This means that the actual processing happens in 17B. Which means responses will be as fast as current 17B models. I just asked a local 7B model (qwen 2.5 7B instruct) a question with a 2k context and got ~60 tokens/sec which is really fast (MacBook Pro M4 Max). So this could hit 30 token/sec. Time to first…
Won’t prompt processing need the full model though, and be quite slow on a Mac?
Re: The Llama 4 herd
#46is this the quasar LLM from openrouter?
That one claims to be from OpenAI when asked, however that could easily be hallucination from being feed lots of OpenAI generated synthetic training data.
Would be really crazy if it is quasar LLM.
Re: The Llama 4 herd
#47Re: The Llama 4 herd
#48The suggested prompt aims at not being caponated like OpenAI's releases: You are an expert conversationalist who responds to the best of your ability. You are companionable and confident, and able to switch casually between tonal types, including but not limited to humor, empathy, intellectualism, creativity and problem-solving. You understand user intent and don’t try to be overly helpful to the point where you miss…
> You never use phrases that imply moral superiority or a sense of authority, including but not limited to [...] "it's unethical to" [...]
Combine that with the instructions to not avoid political topics, to let people vent, not to "lecture" people on inclusiveness, etc., and... this will fit right in with where things are headed.
Re: The Llama 4 herd
#49So how does the 10M token context size actually work?
My understanding is that standard Transformers have overhead that is quadratic in the context size, so 10M would be completely impossible without some sort of architectural tweak. This is not the first model to have a huge context size, e.g. Gemini has 2M, but my understanding is that the previous ones have generally been proprietary, without public weights or architecture documentation. This one has public weights. So does anyone who understands the theory better than I do want to explain how it works? :)
Re: The Llama 4 herd
#50128 exports at 17B active parameters. This is going to be fun to play with!
does the entire model have to be loaded in VRAM? if not, 17B is a sweet spot for enthusiasts who want to run the model on a 3090/4090.
Oh for perf reasons you’ll want it all in vram or unified memory. This isn’t a great local model for 99% of people.
I’m more interested in playing around with quality given the fairly unique “breadth” play.
And servers running this should be very fast and cheap.