Brave Leo now uses Mixtral 8x7B as default

Post by freediver » Sat, Jan 27, 2024, 4:34 PM UTC

mcbuilder wrote: post #40

thebruce87m wrote: post #30

Earlier quoted context omitted.

What is the definition of a submarine comment? Google fails and ChatGPT says: > A "submarine comment" on social media refers to a comment that is made on an old post or thread, long after the conversation has died down. This term derives from the idea of a submarine which remains submerged and out of sight for long periods before suddenly surfacing. In the context of social media, it's when someone delves deep into s…

I think GP is trying to coin a term for stealth marketing Hacker News comments, except the analogy doesn't really make sense.

Post by jodleif » Sat, Jan 27, 2024, 4:37 PM UTC

rhdunn wrote: post #14

jodleif wrote: post #3

Earlier quoted context omitted.

I prefer koboldcpp over llama.cpp. It’s easy to spilt between gpu/cpu on models larger than VRAM

That's interesting. It also looks like koboldcpp works better with long interactions, as it only processes changed tokens. I'm using llama.cpp with text-generation-webui and its OpenAI compatible API. I'll have to look to see if I can use koboldcpp with it.

Post by deely3 » Sat, Jan 27, 2024, 4:50 PM UTC

thebruce87m wrote: post #30

deely3 wrote: post #8

Earlier quoted context omitted.

Is this submarine comment?

What is the definition of a submarine comment? Google fails and ChatGPT says: > A "submarine comment" on social media refers to a comment that is made on an old post or thread, long after the conversation has died down. This term derives from the idea of a submarine which remains submerged and out of sight for long periods before suddenly surfacing. In the context of social media, it's when someone delves deep into s…

Post by [deleted] » Sat, Jan 27, 2024, 5:09 PM UTC

beefield wrote: post #38

beck5 wrote: post #27

Earlier quoted context omitted.

Dumb question, but how can a 32 bit number be converted to 2 bits and still be useful? It seems like magic.

I think of it with this kind of analogy: the original image is stored with 32 bit color scheme. You can reduce the color scheme to 16 bit accuracy and still figure out pretty well what the image is about. 2 bit is stretching this to a bit far, basically either pixel is white or it is black, but even if you lose lots of nuances in the image, in many images even that gives you some idea whats going on in the image.

Post by RupertEisenhart » Sat, Jan 27, 2024, 5:32 PM UTC

bearjaws wrote: post #86

firtoz wrote: post #11

What are good API providers that serve mixtral? I know only octo ai which seems decent but will be good to know alternatives too

Together.ai seems to be the best, incredibly fast.