Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Mixtral 8x22B

mistral.ai

Post reply on HN
Print view View original

201–210 of 252 posts

Re: Mixtral 8x22B

#201

Post by ajcp »

neillyons wrote: post #141
hubraumhugo wrote: post #42

It feels absolutely amazing to build an AI startup right now. It's as if your product automatically becomes cheaper, more reliable, and more scalable with each new major model release. - We first struggled with limited context windows [solved] - We had issues with consistent JSON ouput [solved] - We had rate limiting and performance issues for the large 3rd party models [solved] - Hosting our own OSS models for small…

> We had issues with consistent JSON ouput [solved] It says the JSON output is constrained via their platform (on la Plateforme). Does that mean JSON output is only available in the hosted version? Are there any small models that can be self hosted that output valid JSON.

> Does that mean JSON output is only available in the [self]-hosted version?

I would assume so. They probably constrain JSON output so that the JSON response doesn't bork the front-end/back-end of la Plateforme itself as it moves through their code back to you.

Re: Mixtral 8x22B

#202

Post by zone411 »

arnaudsm wrote: post #10

Curious to see how it performs against GPT-4. Mixtral8x22 beats CommandR+, which is at GPT-4-level in LMSYS' leaderboard.

LMSYS leaderboard is just one benchmark (that I think is fundamentally flawed). GPT-4 is clearly better.

Re: Mixtral 8x22B

#203

Post by zone411 »

It ranks between Mistral Small and Mistral Medium on my NYT Connections benchmark and is indeed better than Command R Plus and Qwen 1.5 Chat 72B, which were the top two open weights models. Grok 1.0 is not an instruct model, so it cannot be compared fairly.

Re: Mixtral 8x22B

#204

Post by qeternity »

orra wrote: post #196

Is this release a pleasant surprise? Mistral weakened their commitment to open source when they partnered with Microsoft. It's nice they're using some of the money from their commercial and proprietary models, to improve the state of the art for open source (open weights) models.

Mistral just released the most powerful open weight model in the history of humanity.

How did they weaken their commitment to open weights?

Re: Mixtral 8x22B

#205

Post by samus »

apexalpha wrote: post #64
Lacerda69 wrote: post #51

Earlier quoted context omitted.

I run it fine on my 64gb RAM beast.

Is that normal ram of GPU ram?

64GB is not GPU RAM, but system RAM. Consumer GPUs have 24GB at most, those with good value/price have way less. Current generation workstation GPUs are unaffordable; used can be found on ebay for a reasonable price, but they are quite slow. DDR5 RAM might be a better investment.

Re: Mixtral 8x22B

#206

Post by qeternity »

kristopolous wrote: post #191
sva_ wrote: post #99

Earlier quoted context omitted.

Plenty of laptops still have SO-DIMM, such as EliteBook for example. People need to vote with their wallet, and not buy stuff that goes against their principles.

There needs to be more fidelity than "vote with wallet". Let's say I decided to not purchase your product. Why? The question remains unanswered. Perhaps I didn't see it for sale or Bob in accounting just got one and I didn't want to look like I was copying Bob. Even at scale this doesn't work. Let's say Lenovo switches to making all of their laptops hot pink with bedazzled rhinestone butterflies and sales plummet. Yo…

You misunderstand the signal. The signal is “you’re doing something wrong”. Companies have tremendous incentive to figure out what that is. They do huge amounts of market research and customer feedback.

Re: Mixtral 8x22B

#208

Post by orra »

qeternity wrote: post #204
orra wrote: post #196

Is this release a pleasant surprise? Mistral weakened their commitment to open source when they partnered with Microsoft. It's nice they're using some of the money from their commercial and proprietary models, to improve the state of the art for open source (open weights) models.

Mistral just released the most powerful open weight model in the history of humanity. How did they weaken their commitment to open weights?

> Mistral just released the most powerful open weight model in the history of humanity.

Well, yeah, it's very welcome, but 'history of humanity' is hyperbole given ChatGPT isn't even two years old.

> How did they weaken their commitment to open weights?

Before https://web.archive.org/web/20240225001133/https://mistral.a... versus after https://web.archive.org/web/20240227025408/https://mistral.a... the Microsoft partnership announcement:

> Committing to open models.

to

> That is why we started our journey by releasing the world’s most capable open-weights models

There were similar changes on their about the Company page.

Re: Mixtral 8x22B

#209

Post by samus »

creshal wrote: post #94
mdrzn wrote: post #14

"64K tokens context window" I do wish they had managed to extend it to at least 128K to match the capabilities of GPT-4 Turbo Maybe this limit will become a joke when looking back? Can you imagine reaching a trillion tokens context window in the future, as Sam speculated on Lex's podcast?

Wasn't there a paper yesterday that turned context evaluation linear (instead of quadratic) and made effectively unlimited context windows possible? Between that and 1.58b quantization I feel like we're overdue for an LLM revolution.

So far, people have come up with many alternatives for quadratic attention. Only recently have they proven their potential.

Re: Mixtral 8x22B

#210

Post by mikegreenberg »

wing-_-nuts wrote: post #145
sleepingreset wrote: post #53

Earlier quoted context omitted.

If you don't mind, I'm trying to experiment w/ local models more. Just now getting into messing w/ these but I'm struggling to come up w/ good use cases. Would you happen to know of any cool OSS model projects that might be good inspiration for a side project? Wondering what most people use these local models for

One idea that I've been mulling over; Given how controllable linux is from the command line, I think it would be somewhat easy to set up a voice to text to a local LLM that could control pretty much everything on command. It would flat out embarass alexa. Imagine 'Hal play a movie', or 'Hal play some music' and it's all running locally, with your content.

There are a few projects doing this. This one piqued my interest as having a potentially nice UX after some maturity. https://github.com/OpenInterpreter/01
Post reply on HN