Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Mixtral 8x22B

mistral.ai

Post reply on HN
Print view View original

21–30 of 252 posts

Re: Mixtral 8x22B

#22

Post by squirrel23 »

dd-dreams wrote: post #2

The development never stops. In a few years we will look back and see how the previous models were and how they're now. How we couldn't run LLaMa 70B on MacBook Air and now we can.

Yes it's pretty cool. There was a neat comparison of deep learning development that I think resonates quite well here.

Around 5 years ago, it took a lambda user some pretty significant hardware, software and time (around a full night), to try to create a short deepfake. Now, you don't need any fancy hardware and you can have some decent results within 5 min on your average computer.

Re: Mixtral 8x22B

#23

Post by htrp »

mdrzn wrote: post #14

"64K tokens context window" I do wish they had managed to extend it to at least 128K to match the capabilities of GPT-4 Turbo Maybe this limit will become a joke when looking back? Can you imagine reaching a trillion tokens context window in the future, as Sam speculated on Lex's podcast?

maybe we'll look back at token context windows like we look back at how much ram we have in a system.

Re: Mixtral 8x22B

#24

Post by clementmas »

I'm considering switching my function calling requests from OpenAI's API to Mistral. Are they using similar formats? What's the easiest way to use Mistral? Is it by using Huggingface?

Re: Mixtral 8x22B

#25

Post by htrp »

kiney wrote: post #17
generalizations wrote: post #4

Earlier quoted context omitted.

...And a massive pile of cash/compute hardware.

not that massive, we're talking six figures. There was a blogpost about this a while back on the startpage of HN.

for finetuning or parameter training from scratch?

Re: Mixtral 8x22B

#26

Post by tartrate »

brokensegue wrote: post #12

Isn't equating active parameters with cost a little unfair since you still need full memory for all the inactive parameters?

Well, since it affects inference speed it means you can handle more in less time, needing less concurrency.

Re: Mixtral 8x22B

#27

Post by jjice »

Does anyone have a good layman's explanation of the "Mixture-of-Experts" concept? I think I understand the idea of having "sub-experts", but how do you decide what each specialization is during training? Or is that not how it works at all?

Re: Mixtral 8x22B

#29

Post by CharlieDigital »

apetresc wrote: post #16
squirrel23 wrote: post #11

Earlier quoted context omitted.

What do you mean?

Virtually every announcement of a new model release has some sort of table or graph matching it up against a bunch of other models on various benchmarks, and they're always selected in such a way that the newly-released model dominates along several axes. It turns interpreting the results into an exercise in detecting which models and benchmarks were omitted.

It would make sense, wouldn't it? Just as we've seen rising fuel efficiency, safety, dependability, etc. over the lifecycle of a particular car model.

The different teams are learning from each other and pushing boundaries; there's virtually no reason for any of the teams to release a model or product that is somehow inferior to a prior one (unless it had some secondary attribute such as requiring lower end hardware).

We're simply not seeing the ones that came up short; we don't even see the ones where it fell short of current benchmarks because they're not worth releasing to the public.

Re: Mixtral 8x22B

#30

Post by 2Gkashmiri »

noman-land wrote: post #20

I'm really excited about this model. Just need someone to quantize it to ~3 bits so it'll run on a 64GB MacBook Pro. I've gotten a lot of use from the 8x7b model. Paired with llamafile and it's just so good.

Can you explain your use case? I tried to get into offline llms, on my machine and even android but without discrete graphics, its a slow hog so I didnt enjoy it but suppose I buy one, what then ?
Post reply on HN