Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Mixtral 8x22B

mistral.ai

Post reply on HN
Print view View original

131–140 of 252 posts

Re: Mixtral 8x22B

#131

Post by jmorgan »

byteknight wrote: post #113

First test I tried to run a random taxation question through it Output: https://gist.github.com/IAmStoxe/7fb224225ff13b1902b6d172467... Within the first paragraph, it outputs: > GET AN ESSAY WRITTEN FOR YOU FROM AS LOW AS $13/PAGE Thought that was hilarious.

The `mixtral:8x22b` tag still points to the text completion model – instruct is on the way, sorry!

Update: mixtral:8x22b now points to the instruct model:

  ollama pull mixtral:8x22b
  ollama run mixtral:8x22b

Re: Mixtral 8x22B

#132

Post by Arcuru »

pants2 wrote: post #116
ru552 wrote: post #33

Earlier quoted context omitted.

easiest is probably with ollama [0]. I think the ollama API is OpenAI compatible. [0] https://ollama.com/

Ollama runs locally. What's the best option for calling the new Mixtral model on someone else's server programmatically?

Openrouter lists several options: https://openrouter.ai/models/mistralai/mixtral-8x22b

Re: Mixtral 8x22B

#133

Post by byteknight »

orost wrote: post #129
byteknight wrote: post #125

Earlier quoted context omitted.

I absolutely did not: ollama run mixtral:8x22b EDIT: I like how you ninja-editted your comment ;)

Considering "mixtral:8x22b" on ollama was last updated yesterday, and Mixtral-8x22B-Instruct-v0.1 (the topic of this post) was released about 2 hours ago, they are not the same model.

Are we looking at the same page?

https://imgur.com/a/y6XfpBl

And even the direct tag page: https://ollama.com/library/mixtral:8x22b shows 40-something minutes ago: https://imgur.com/a/WNhv70B

Re: Mixtral 8x22B

#134

Post by jasonjmcghee »

hubraumhugo wrote: post #42

It feels absolutely amazing to build an AI startup right now. It's as if your product automatically becomes cheaper, more reliable, and more scalable with each new major model release. - We first struggled with limited context windows [solved] - We had issues with consistent JSON ouput [solved] - We had rate limiting and performance issues for the large 3rd party models [solved] - Hosting our own OSS models for small…

How are you approaching hosting? vLLM?

Re: Mixtral 8x22B

#135

Post by vineyardmike »

jjice wrote: post #27

Does anyone have a good layman's explanation of the "Mixture-of-Experts" concept? I think I understand the idea of having "sub-experts", but how do you decide what each specialization is during training? Or is that not how it works at all?

It’s not “experts” in the typical sense of the word. There is no discrete training to learn a particular skill in one expert. It’s more closely modeled as a bunch of smaller models grafted together.

These models are actually a collection of weights for different parts of the system. It’s not “one” neural network. Transformers are composed of layers of transformations to the input, and each step can have its own set of weights. There was a recent video on the front page that had a good introduction to this. There is the MLP, there are the attention heads, etc.

With that in mind, a MoE model is basically where one of those layers has X different versions of the weights, and then an added layer (another neural network with its own weights) that picks the version of “expert” weights to use.

Re: Mixtral 8x22B

#136

Post by orost »

byteknight wrote: post #133
orost wrote: post #129

Earlier quoted context omitted.

Considering "mixtral:8x22b" on ollama was last updated yesterday, and Mixtral-8x22B-Instruct-v0.1 (the topic of this post) was released about 2 hours ago, they are not the same model.

Are we looking at the same page? https://imgur.com/a/y6XfpBl And even the direct tag page: https://ollama.com/library/mixtral:8x22b shows 40-something minutes ago: https://imgur.com/a/WNhv70B

Let me clarify.

Mixtral-8x22B-v0.1 was released a couple days ago. The "mixtral:8x22b" tag on ollama currently refers to it, so it's what you got when you did "ollama run mixtral:8x22b". It's a base model only capable of text completion, not any other tasks, which is why you got a terrible result when you gave it instructions.

Mixtral-8x22B-Instruct-v0.1 is an instruction-following model based on Mixtral-8x22B-v0.1. It was released two hours ago and it's what this post is about.

(The last updated 44 minutes ago refers to the entire "mixtral" collection.)

Re: Mixtral 8x22B

#137

Post by belter »

byteknight wrote: post #133
orost wrote: post #129

Earlier quoted context omitted.

Considering "mixtral:8x22b" on ollama was last updated yesterday, and Mixtral-8x22B-Instruct-v0.1 (the topic of this post) was released about 2 hours ago, they are not the same model.

Are we looking at the same page? https://imgur.com/a/y6XfpBl And even the direct tag page: https://ollama.com/library/mixtral:8x22b shows 40-something minutes ago: https://imgur.com/a/WNhv70B

I get:

ollama run mixtral:8x22b

Error: exception create_tensor: tensor 'blk.0.ffn_gate.0.weight' not found

Re: Mixtral 8x22B

#138

Post by gliptic »

byteknight wrote: post #133
orost wrote: post #129

Earlier quoted context omitted.

Considering "mixtral:8x22b" on ollama was last updated yesterday, and Mixtral-8x22B-Instruct-v0.1 (the topic of this post) was released about 2 hours ago, they are not the same model.

Are we looking at the same page? https://imgur.com/a/y6XfpBl And even the direct tag page: https://ollama.com/library/mixtral:8x22b shows 40-something minutes ago: https://imgur.com/a/WNhv70B

And where does it say that's the instruct model?

Re: Mixtral 8x22B

#139

Post by mysteria »

orost wrote: post #119
byteknight wrote: post #113

First test I tried to run a random taxation question through it Output: https://gist.github.com/IAmStoxe/7fb224225ff13b1902b6d172467... Within the first paragraph, it outputs: > GET AN ESSAY WRITTEN FOR YOU FROM AS LOW AS $13/PAGE Thought that was hilarious.

That's not the model this post is about. You used the base model, not trained for tasks. (The instruct model is probably not on ollama yet.)

Yeah this is exactly what happens when you ask a base model a question. It'll just attempt to continue what you already wrote based off its training set, so if you say have it continue a story you've written it may wrap up the story and then ask you to subscribe for part 2, followed by a bunch of social media comments with reviews.

Re: Mixtral 8x22B

#140

Post by GaggiX »

kristianp wrote: post #114

So this one is 3x the size but only 7% better on MMLU? Given Moores law is mostly dead, this trend is going to make for even more extremely expensive compute for next gen AI models.

That's 25% fewer errors.
Post reply on HN