351–360 of 482 posts
How come the answers for Mixtral 8x7B-32k and Llama 2 70B-4k are identical? After asking via Mixtral a couple of questions I switched to Llama, and while it shows Llama as the Model used for the response, the answer is identical. See first and last question: https://pastebin.com/ZQV10C8Q
How come the answers for Mixtral 8x7B-32k and Llama 2 70B-4k are identical? After asking via Mixtral a couple of questions I switched to Llama, and while it shows Llama as the Model used for the response, the answer is identical. See first and last question: https://pastebin.com/ZQV10C8Q
Yeah, it's confusing. See here for an explanation: https://news.ycombinator.com/item?id=39431921
How come the answers for Mixtral 8x7B-32k and Llama 2 70B-4k are identical? After asking via Mixtral a couple of questions I switched to Llama, and while it shows Llama as the Model used for the response, the answer is identical. See first and last question: https://pastebin.com/ZQV10C8Q
Yeah, it's confusing. See here for an explanation: https://news.ycombinator.com/item?id=39431921
Earlier quoted context omitted.
You also have to be an insider. If I go to a bar, and overhear a pair of Googlers discussing something secret and overhear it, I can: 1) Trade on it. 2) Talk about it. Because I'm not an insider. On the other hand, if I'm sleeping with the CEO, I become an insider. Not a lawyer. Above is not legal advice. Just a comment that the line is much more complex, and talking about a potential acquisition is usually okay (if…
It doesn't matter if you overheard it at a bar or if you're just some HN commenter posting completely incorrect legal advice; the law prohibits trading on material nonpublic information. I would pay a lot to see you try your ridiculous legal hokey-pokey on how to define an "insider."
Earlier quoted context omitted.
It doesn't matter if you overheard it at a bar or if you're just some HN commenter posting completely incorrect legal advice; the law prohibits trading on material nonpublic information. I would pay a lot to see you try your ridiculous legal hokey-pokey on how to define an "insider."
If you did hear it in a bar, could you tweet it out before your trade, so the information is made public?
Impressive demo! However, the hardware requirements and cost make this inaccessible for anyone but large companies. When do you envision that the price could be affordable for hobbyists? Also, while the CNN Vapi demo was impressive as well, a few weeks ago here[1] someone shared https://smarterchild.chat/ . That also has _very_ low audio latency, making natural conversation possible. From that discussion it seems tha…
> However, the hardware requirements and cost make this inaccessible for anyone but large companies. When do you envision that the price could be affordable for hobbyists? For API access to our tokens as a service we guarantee to beat any other provider on cost per token (see https://wow.groq.com ). In terms of selling hardware, we're focused on selling whole systems, and they're only really suitable for corporations…
In the demo alone I just used way more tokens than I normally would testing an LLM since it was so amazingly fast.
Earlier quoted context omitted.
What do you mean by non-deterministic hardware? cuBLAS on a laptop GPU was deterministic when I tried it last iirc
Tip of the ice-berg. DRAM needs to be refreshed every X cycles. This means you don't know the time it takes to read from memory. You could be reading at a refresh cycle. This circuitry also adds latency.
Earlier quoted context omitted.
Groq Engineer here, I'm not seeing why being able to scale compute outside of a single card/node is somehow a problem. My preferred analogy is to a car factory: Yes, you could build a car with say only one or two drills, but a modern automated factory has hundreds of drills! With a single drill, you could probably build all sorts of cars, but a factory assembly line is only able to make specific cars in that configur…
> 30b+ parameter model doing RAG as part of a conversation with voice responses in less than a second, running on Nvidia. I believe that this is doable - my pipeline is generally closer to 400ms without RAG and with Mixtral, with a lot of non-ML hacks to get there. It would also definitely be doable with a joint speech-language model that removes the transcription step. For these use cases, time to first byte is the…
The most interesting applications of LLMs are not chatbots.