Llama3 on Groq

#1

Post by matanyall » Fri, Apr 19, 2024, 7:04 PM UTC

Re: Llama3 on Groq

#2

Post by Oras » Fri, Apr 19, 2024, 7:10 PM UTC

That's impressive. I asked to summarise an article in 5 bullet points, and the output was 812.81 T/s on Llama 3 8B.

#3

Post by [deleted] » Fri, Apr 19, 2024, 7:28 PM UTC

[deleted]

#4

Post by WhatsName » Fri, Apr 19, 2024, 8:55 PM UTC

What is tbe cost per Mio. Token for llama3 70b on groq?

#5

Post by jacooper » Fri, Apr 19, 2024, 9:06 PM UTC

When is Mixtral 8x22b coming?

#6

Post by matanyall » Fri, Apr 19, 2024, 10:02 PM UTC

WhatsName wrote: post #4

What is tbe cost per Mio. Token for llama3 70b on groq?

$0.59 input, $0.79 output (https://wow.groq.com/)

#7

Post by frozenport » Sat, Apr 20, 2024, 1:48 AM UTC

LLama3 looks particularly good at tool calling

Groq's low latency is particularly good for tool calling

Seems like two techs that will make coding obsolete :-)

#8

Post by Alifatisk » Sat, Apr 20, 2024, 10:13 AM UTC

Is the python lib open-source? I could only find the ja lib for Groq.

#9

Post by Alifatisk » Sat, Apr 20, 2024, 12:38 PM UTC

Alifatisk wrote: post #8

Is the python lib open-source? I could only find the ja lib for Groq.