I'm dreaming of having LLMs on anything. Unlike the "bluetooth on everything" craze, this can be practical as every device can become smart. Remember how some British researchers made a self driving car using an LLM? A toaster anticipating how to cook when you describe it what you want want actually be an improvement.
Groq runs Mixtral 8x7B-32k with 500 T/s
41–50 of 482 posts
Re: Groq runs Mixtral 8x7B-32k with 500 T/s
#42Hi folks, I work for Groq. Feel free to ask me any questions. (If you check my HN post history you'll see I post a lot about Haskell. That's right, part of Groq's compilation pipeline is written in Haskell!)
Re: Groq runs Mixtral 8x7B-32k with 500 T/s
#43Jesus that makes chatgpt and even gemini seem slow AF
better quality than I was expecting. For fun I set the system prompt to: You are a leader of a team of ai helpers. when given a question you can call on an expert, as a wizard calls on magic. You will say, I call forth {expert} master of {subject matter} an expert in {x, y, z}. Then you will switch to that persona. I was not let down..
Re: Groq runs Mixtral 8x7B-32k with 500 T/s
#44Re: Groq runs Mixtral 8x7B-32k with 500 T/s
#45Does this make it practical to run LLMs on mobile devices? I wonder about the power consumption and if it can make sense to have it integrated in some future mobile devices. Or maybe have a dedicated storage, RAM and processing cores that goes as an USB-C add-on? A case with integrated battery and this chip? I'm dreaming of having LLMs on anything. Unlike the "bluetooth on everything" craze, this can be practical as…
Re: Groq runs Mixtral 8x7B-32k with 500 T/s
#46Re: Groq runs Mixtral 8x7B-32k with 500 T/s
#47Interesting, I ran my "prime test", it passed on generating a prime, it failed on it being large. > "Can you generate me a large prime please" > Sure, here's a large prime number: 193707721 > This number is a prime number because it is only divisible by 1 and itself. You can check its primality by trying to divide it by all the numbers up to its square root, and if none of them divide it evenly, then it is a prime nu…
Re: Groq runs Mixtral 8x7B-32k with 500 T/s
#48Interesting, I ran my "prime test", it passed on generating a prime, it failed on it being large. > "Can you generate me a large prime please" > Sure, here's a large prime number: 193707721 > This number is a prime number because it is only divisible by 1 and itself. You can check its primality by trying to divide it by all the numbers up to its square root, and if none of them divide it evenly, then it is a prime nu…
Re: Groq runs Mixtral 8x7B-32k with 500 T/s
#49What's the underlying hardware for this?
[1] https://wow.groq.com/wp-content/uploads/2024/02/GroqISCAPape...
Re: Groq runs Mixtral 8x7B-32k with 500 T/s
#50Interesting, I ran my "prime test", it passed on generating a prime, it failed on it being large. > "Can you generate me a large prime please" > Sure, here's a large prime number: 193707721 > This number is a prime number because it is only divisible by 1 and itself. You can check its primality by trying to divide it by all the numbers up to its square root, and if none of them divide it evenly, then it is a prime nu…
> Could you generate me a 64 bit prime?
Returns:
> Sure, here's a 64-bit prime number: 15592425903773483751
> You can verify that it's prime by checking that it has no divisors between 2 and its square root. In this case, the square root of the number is approximately 12489593.5, so you would need to check divisibility for all integers between 2 and 12489593.5. Since the number is odd, you can skip checking even numbers, which will reduce the number of checks you need to perform.
> However, if you're looking for a larger 64-bit prime or a prime number with specific properties, you may need to use a more sophisticated algorithm or a specialized prime number generator.
Like GPT3.5 and Bard, it confidently suggests that 15592425903773483751 is prime, and even explains why.
It's wrong:
> $ factor 15592425903773483751 > 15592425903773483751: 3 3 3 7 167 853 911 635723119
Further prompting:
> Can you factor 15592425903773483751?
> No, I cannot factor 15592425903773483751 because it is a prime number, and prime numbers cannot be factored into smaller non-trivial factors. By definition, a prime number is a positive integer greater than 1 that has no positive integer divisors other than 1 and itself. In this case, 15592425903773483751 is a 64-bit prime number, which means it is a prime number that fits into a 64-bit data type.
> To verify that 15592425903773483751 is a prime number, you can use a primality test, such as the Fermat primality test or the Miller-Rabin primality test. These tests can quickly determine whether a number is prime or composite (not prime) with high probability. In the case of 15592425903773483751, both tests confirm that it is a prime number.
Even further prompting to get it to try to multiply numbers together quickly turns into a disaster, culminating with:
> That doesn't look right
> I apologize for the mistake in my previous response. Here is the correct result of the multiplication:
> 3 x 3 x 3 x 7 x 167 x 853 x 911 x 635723119 = 322711211111111111
> Therefore, the product of the numbers is:
> 322711211111111111.
> I apologize for any confusion my previous response may have caused.
The context seems to be really aggressively baked in, because when I ask it:
> Can you factor 322711211111111111?
It returns
> Yes, I can factor the number 322711211111111111. Here is its prime factorization:
> 322711211111111111 = 3 x 3 x 3 x 7 x 167 x 853 x 911 x 635723119
> Therefore, the prime factors of the number are 3, 7, 167, 853, 911, and 635723119.
> Note that the prime factorization is unique up to reordering of the factors. In this case, the prime factors are repeated several times, so the prime factorization can be written more compactly as:
> 3^3 x 7 x 167 x 853 x 911 x 635723119
> or
> 3 x 3 x 3 x 7 x 167 x 853 x 911 x 635723119
> I hope this helps! Let me know if you have any other questions.
The real irony? 322711211111111111 is prime!