Mistral AI Launches New 8x22B MOE Model

Post by Eisenstein » Thu, Apr 11, 2024, 5:35 PM UTC

hmottestad wrote: post #77

nazka wrote: post #52

Out of topic but are we now back at the same performance than ChatGPT 4 at the time people said it worked like magic (meaning before the nerf to make it more politically correct but making his performance crash)?

I’ve been testing a lot of LLMs on my MacBook and I would say that all of them are far away from being as good as GPT-4, at any time. Many are as good as GPT-3 though. There are also a lot of models that are fine tuned for specific tasks. Language support is one big thing that is missing from open models. I’ve only found one model that can do anything useful with Norwegian, which has never been an issue GPT-4.

Post by bevekspldnw » Thu, Apr 11, 2024, 6:20 PM UTC

mirekrusin wrote: post #139

GuB-42 wrote: post #127

Earlier quoted context omitted.

It is there, not for all the benchmarks, but for those where it is included, GPT-4 scores much higher. Not surprising since GPT-4 is still state-of-the-art and much bigger. Where Mistral has been particularly impressive is when you take the size of the model into account.

GPT-4 is instruct tuned model, of course it's going to score higher, apples and oranges.

Post by bevekspldnw » Thu, Apr 11, 2024, 6:40 PM UTC

MPSimmons wrote: post #138

bevekspldnw wrote: post #118

Earlier quoted context omitted.

There is a user called The Bloke on hugging face- they release pre quantized models pretty soon after the full size drop. Just watch their page and pray you can fit the 4 bit in your GPU. I’m sure they are already working on it.

I think 4b for this is support to be over 70GB, so definitely still heavy hardware.

Post by fzzzy » Thu, Apr 11, 2024, 7:38 PM UTC

bevekspldnw wrote: post #144

nathanasmith wrote: post #133

Earlier quoted context omitted.

TheBloke stopped uploading in January. There are others that have stepped up though.

Oh really? Who else should I be looking at? That person is a hero, super bummed!

Post by hedgehog » Thu, Apr 11, 2024, 8:36 PM UTC

cjbprime wrote: post #105

hedgehog wrote: post #86

Earlier quoted context omitted.

I'm curious how the newer consumer Ryzens might fare. With LPDDR5X they have >100 GB/s memory bandwidth and the GPUs have been improved quite a bit (16 TFLOPS FP16 nominal in the 780M). There are likely all kinds of software problems but setting that aside the perf/$ and perf/watt might be decent.

Consumer Ryzens only have two-channel memory controllers. Two dual-rank (double sided) DIMMs per channel, which you would need to use to get enough RAM for LLMs, drops the memory bandwidth dramatically -- almost all the way back down to DDR4 speeds.