Mixtral 8x22B • Hacker News Board

Re: Mixtral 8x22B

#141

Post by neillyons » Wed, Apr 17, 2024, 4:03 PM UTC

hubraumhugo wrote: post #42

It feels absolutely amazing to build an AI startup right now. It's as if your product automatically becomes cheaper, more reliable, and more scalable with each new major model release. - We first struggled with limited context windows [solved] - We had issues with consistent JSON ouput [solved] - We had rate limiting and performance issues for the large 3rd party models [solved] - Hosting our own OSS models for small…

> We had issues with consistent JSON ouput [solved]

It says the JSON output is constrained via their platform (on la Plateforme).

Does that mean JSON output is only available in the hosted version? Are there any small models that can be self hosted that output valid JSON.

Re: Mixtral 8x22B

#142

Post by renewiltord » Wed, Apr 17, 2024, 4:04 PM UTC

byteknight wrote: post #113

First test I tried to run a random taxation question through it Output: https://gist.github.com/IAmStoxe/7fb224225ff13b1902b6d172467... Within the first paragraph, it outputs: > GET AN ESSAY WRITTEN FOR YOU FROM AS LOW AS $13/PAGE Thought that was hilarious.

Not instruct tuned. You're (actually) "holding it wrong".

Re: Mixtral 8x22B

#143

Post by jhoechtl » Wed, Apr 17, 2024, 4:05 PM UTC

Did anyone have success getting danswer and ollama to work together?

Re: Mixtral 8x22B

#144

Post by Me1000 » Wed, Apr 17, 2024, 4:06 PM UTC

belter wrote: post #137

byteknight wrote: post #133

Earlier quoted context omitted.

Are we looking at the same page? https://imgur.com/a/y6XfpBl And even the direct tag page: https://ollama.com/library/mixtral:8x22b shows 40-something minutes ago: https://imgur.com/a/WNhv70B

I get: ollama run mixtral:8x22b Error: exception create_tensor: tensor 'blk.0.ffn_gate.0.weight' not found

You need to update ollama to 0.1.32.

Re: Mixtral 8x22B

#145

Post by wing-_-nuts » Wed, Apr 17, 2024, 4:08 PM UTC

sleepingreset wrote: post #53

hubraumhugo wrote: post #42

It feels absolutely amazing to build an AI startup right now. It's as if your product automatically becomes cheaper, more reliable, and more scalable with each new major model release. - We first struggled with limited context windows [solved] - We had issues with consistent JSON ouput [solved] - We had rate limiting and performance issues for the large 3rd party models [solved] - Hosting our own OSS models for small…

If you don't mind, I'm trying to experiment w/ local models more. Just now getting into messing w/ these but I'm struggling to come up w/ good use cases. Would you happen to know of any cool OSS model projects that might be good inspiration for a side project? Wondering what most people use these local models for

One idea that I've been mulling over; Given how controllable linux is from the command line, I think it would be somewhat easy to set up a voice to text to a local LLM that could control pretty much everything on command.

It would flat out embarass alexa. Imagine 'Hal play a movie', or 'Hal play some music' and it's all running locally, with your content.

Re: Mixtral 8x22B

#146

Post by nicklecompte » Wed, Apr 17, 2024, 4:10 PM UTC

Lacerda69 wrote: post #57

imjonse wrote: post #40

Earlier quoted context omitted.

Calling the model 'truly open' without is not technically correct though.

It's open enough for all practical purposes IMO.

It's not "open enough" to do an honest evaluation of these systems by constructing adversarial benchmarks.

Re: Mixtral 8x22B

#147

Post by ado__dev » Wed, Apr 17, 2024, 4:11 PM UTC

We rolled out Mixtral 8x22b to our LLM Litmus Test at s0.dev for Cody AI. Don't have enough data to say it's better or worse that other LLMs yet, but if you want to try it out for coding purposes, let me know your experience.

Re: Mixtral 8x22B

#148

Post by cptcobalt » Wed, Apr 17, 2024, 4:12 PM UTC

kaibee wrote: post #107

kiney wrote: post #48

Earlier quoted context omitted.

from scratch: https://research.myshell.ai/jetmoe

That's for an 8B model.

This is over trivializing it, but there isn't much more inherent complexity in training an 8B or larger model other than more money, more compute, more data, more time. Overall, the principles are similar.

Re: Mixtral 8x22B

#149

Post by imjonse » Wed, Apr 17, 2024, 4:12 PM UTC

Lacerda69 wrote: post #57

imjonse wrote: post #40

Earlier quoted context omitted.

Calling the model 'truly open' without is not technically correct though.

It's open enough for all practical purposes IMO.

As open as an executable binary that you are allowed to download and use for free.

Re: Mixtral 8x22B

#150

Post by azinman2 » Wed, Apr 17, 2024, 4:13 PM UTC

squirrel23 wrote: post #22

dd-dreams wrote: post #2

The development never stops. In a few years we will look back and see how the previous models were and how they're now. How we couldn't run LLaMa 70B on MacBook Air and now we can.

Yes it's pretty cool. There was a neat comparison of deep learning development that I think resonates quite well here. Around 5 years ago, it took a lambda user some pretty significant hardware, software and time (around a full night), to try to create a short deepfake. Now, you don't need any fancy hardware and you can have some decent results within 5 min on your average computer.

That part isn’t very good.

https://www.nytimes.com/2024/04/08/technology/deepfake-ai-nu...