41–50 of 279 posts
Statement from Mark: https://about.fb.com/news/2024/07/open-source-ai-is-the-path...
Earlier quoted context omitted.
Your going to need a lot more than a few, 800G VRAM needed
how is this even useful? no one can run it.
If you are running a commercial service that uses AI, you buy a few dozen A100s, spend a half million, and you are good for a while.
If you are running a commercial inferencing service, you spend tens of millions or get a cloud sponsor.
Open source models are very exciting for self hosting, but the per-token hosted inference pricing hasn't been competitive with OpenAI and Anthropic, at least for a given tier of quality. (E.g.: Llama 3 70B costing between $1 and $10 per million tokens on various platforms, but Claude Sonnet 3.5 is $3 per million.)
Today appears to be the day you can run an LLM that is competitive with GPT-4o at home with the right hardware. Incredible for progress and advancement of the technology. Statement from Mark: https://about.fb.com/news/2024/07/open-source-ai-is-the-path...
Where the right hardware is 10x4090s even at 4 bits quantization. I'm hoping we'll see these models get smaller, but the GPT-4-competitive one isn't really accessible for home use yet.
Still amazing that it's available at all, of course!
I wrote about this when llama-3 came out, and this launch confirms it: Meta's goal from the start was to target OpenAI and the other proprietary model players with a "scorched earth" approach by releasing powerful open models to disrupt the competitive landscape. Meta can likely outspend any other AI lab on compute and talent: - OpenAI makes an estimated revenue of $2B and is likely unprofitable. Meta generated a rev…
Today appears to be the day you can run an LLM that is competitive with GPT-4o at home with the right hardware. Incredible for progress and advancement of the technology. Statement from Mark: https://about.fb.com/news/2024/07/open-source-ai-is-the-path...
Does anyone know why they haven't released any 30B-ish param models? I was expecting that to happen with this release and have been disappointed once more. They also skipped doing a 30B-ish param model for llama2 despite claiming to have trained one.
I wrote about this when llama-3 came out, and this launch confirms it: Meta's goal from the start was to target OpenAI and the other proprietary model players with a "scorched earth" approach by releasing powerful open models to disrupt the competitive landscape. Meta can likely outspend any other AI lab on compute and talent: - OpenAI makes an estimated revenue of $2B and is likely unprofitable. Meta generated a rev…
This is very impressive, though an adjacent question — does anyone know roughly how much time and compute cost it takes to train something like the 405B? I would imagine with all the compute Meta has that the moat is incredibly large in terms of being able to train multiple 405B-level morels and compete.
https://github.com/meta-llama/llama-models/blob/main/models/...