Brave Leo now uses Mixtral 8x7B as default • Hacker News Board

Re: Brave Leo now uses Mixtral 8x7B as default

#181

Post by tome » Wed, Jan 31, 2024, 10:54 AM UTC

attentive wrote: post #151

tacoman wrote: post #95

Earlier quoted context omitted.

I am using the exact same model. Ryzen 5600G w/32GB and an Nvidia P40 w/24GB VRAM 20/33 layers offloaded to GPU, 4K context. Uses 25GB system RAM and all 24GB VRAM. 5-7 tokens per second.

and Groq does 485.08 T/s on mixtral 8x7B-32k I am not sure local models have any future other than POC/research. Depends on the cost of course.

(Groqster here) For anyone who wants to try it, you can go to https://chat.groq.com/ and choose Mixtral from the drop-down menu. Also, feel free to ask me any questions about Groq hardware or service.

Re: Brave Leo now uses Mixtral 8x7B as default

#182

Post by tome » Wed, Jan 31, 2024, 10:55 AM UTC

frozenport wrote: post #178

attentive wrote: post #152

Earlier quoted context omitted.

I just discovered Groq, which does 485.08 T/s on mixtral 8x7B-32k No idea on pricing but supposedly one can email to api@groq.com

I think you can try it online at chat.groq.com

(Groqster here). Yes, you can select Mixtral from the dropdown menu. If anyone has any questions about Groq let me know and I'll do my best to answer!

Re: Brave Leo now uses Mixtral 8x7B as default

#183

Post by tome » Wed, Jan 31, 2024, 11:02 AM UTC

firtoz wrote: post #11

What are good API providers that serve mixtral? I know only octo ai which seems decent but will be good to know alternatives too

I work for Groq and we serve the fastest available version of Mixtral (by far) and we also have a web chat app. I'll refrain from linking it because it has already been linked and I don't want to spam, but I'm available to answer any questions people have about Groq's hardware and service.

Re: Brave Leo now uses Mixtral 8x7B as default

#184

Post by soundnote » Wed, Feb 07, 2024, 3:54 PM UTC

Diti wrote: post #85

davikr wrote: post #4

It's nice using Brave because you have Chromium's better performance, without having to worry about Manifest V2 dying and taking adblocking down with it. I have uBlock Origin enabled, but it has barely caught anything that slipped past the browser filters.

If by performance you mean browser performance, you have more performance with Firefox nowadays. https://news.ycombinator.com/item?id=36770883

One genuine performance edge Firefox has is that it can handle giant YouTube comments sections better than Chromium browsers in part because it accepts larger maximum page size.