cerebras: 450 tokens/sec llama 3.1 70B
theregister.com
cerebras: 450 tokens/sec llama 3.1 70B
1–3 of 3 posts
Re: cerebras: 450 tokens/sec llama 3.1 70B
#2- 1,800tps on llama 3.1 8B
- 450tps on llama 3.1 70B
free chat interface is at: https://inference.cerebras.ai (requires login)
Re: cerebras: 450 tokens/sec llama 3.1 70B
#3Cerebras fails the "how many r's in strawberry" test.
Grok is the only one who passed that test.
Going to be interesting to see the speed and accuracy keep increasing, cant imagine how fast/accurate things will be in a decade. Cant wait.