151–160 of 279 posts
You can chat with these new models at ultra-low latency at groq.com. 8B and 70B API access is available at console.groq.com. 405B API access for select customers only – GA and 3rd party speed benchmarks soon. If you want to learn more, there is a writeup at https://wow.groq.com/now-available-on-groq-the-largest-and-m... . (disclaimer, I am a Groq employee)
405B is already being served on WhatsApp! https://ibb.co/kQ2tKX5
Earlier quoted context omitted.
> specifically, it puts restrictions on commercial use for some users (paragraph 2) and also restricts the use of the model and software for certain purposes (the Acceptable Use Policy) It's "a Google and Apple can't use this model in production" clause that frankly we can all be relatively okay with.
but it means your company cant be acquired by those giants, if you use this model.
You're only ok with it if you're not interested in having maximum freedom of movement vis-a-vis any potential exits.
The 405b model is actually competitive against closed source frontier models. Quick comparison with GPT-4o: +----------------+-------+-------+ | Metric | GPT-4o| Llama | | | | 3.1 | | | | 405B | +----------------+-------+-------+ | MMLU | 88.7 | 88.6 | | GPQA | 53.6 | 51.1 | | MATH | 76.6 | 73.8 | | HumanEval | 90.2 | 89.0 | | MGSM | 90.5 | 91.6 | +----------------+-------+-------+
This nodel is not “open source”, free to use maybe.
Earlier quoted context omitted.
So are they actually making the models open now or are they staying the course with "kind of open" as they have done for LLaMA 1, 2, and 3 [1]? [1]: https://opensource.org/blog/metas-llama-2-license-is-not-ope... As I have stated time and again, it is perfectly fine for them to slap on whatever license they see fit as it is their work. But it would be nice if they used appropriate terms so as not to disrupt the disco…
> specifically, it puts restrictions on commercial use for some users (paragraph 2) and also restricts the use of the model and software for certain purposes (the Acceptable Use Policy) It's "a Google and Apple can't use this model in production" clause that frankly we can all be relatively okay with.
Earlier quoted context omitted.
You can build a machine that will run the 405b model for much, much less, if you're willing to accept the following caveats: * You'll be running a Q5(ish) quantized model, not the full model * You're OK with buying used hardware * You have two separate 120v circuits available to plug it into (I assume you're in the US), or alternatively a single 240v dryer/oven/RV-style plug. The build would look something like (appr…
Would be interesting to see the performance on a dual-socket EPYC system with DDR5 running at maximum speed. Assuming NUMA doesn't give you headaches (which it will) you would be looking at nearly 1 TB/s
@dang why was this removed/filtered from the front page?
GPT-4o 30.7
GPT-4 turbo (2024-04-09) 29.7
Llama 3.1 405B Instruct 29.5
Claude 3.5 Sonnet 27.9
Claude 3 Opus 27.3
Llama 3.1 70B Instruct 26.4
Gemini Pro 1.5 0514 22.3
Gemma 2 27B Instruct 21.2
Mistral Large 17.7
Gemma 2 9B Instruct 16.3
Qwen 2 Instruct 72B 15.6
Gemini 1.5 Flash 15.3
GPT-4o mini 14.3
Llama 3.1 8B Instruct 14.0
DeepSeek-V2 Chat 236B (0628) 13.4
Nemotron-4 340B 12.7
Mixtral-8x22B Instruct 12.2
Yi Large 12.1
Command R Plus 11.1
Mistral Small 9.3
Reka Core-20240501 9.1
GLM-4 9.0
Qwen 1.5 Chat 32B 8.7
Phi-3 Small 8k 8.4
DBRX 8.0
I wrote about this when llama-3 came out, and this launch confirms it: Meta's goal from the start was to target OpenAI and the other proprietary model players with a "scorched earth" approach by releasing powerful open models to disrupt the competitive landscape. Meta can likely outspend any other AI lab on compute and talent: - OpenAI makes an estimated revenue of $2B and is likely unprofitable. Meta generated a rev…
> Open source likely attracts better talent and researchers I work at OpenAI and used to work at meta. Almost every person from meta that I know has asked me for a referral to OpenAI. I don’t know anyone who left OpenAI to go to meta.
Earlier quoted context omitted.
This nodel is not “open source”, free to use maybe.
I really wish people would use "open weights" rather than "open source". It's precise and obvious, and leaves an accurate descriptor for actual "open source" models, where the source and methods that that generate the artifact , that is the weights, is open.