Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Mistral-8x7B-Chat

huggingface.co

Post reply on HN
Print view View original

61–70 of 75 posts

Re: Mistral-8x7B-Chat

#61

Post by stavros »

gregsadetsky wrote: post #45
lannisterstark wrote: post #5

Man this LLM stuff gets released faster than I can keep up. Is there a centralized list somewhere that tests "use this for x purpose, use that for y?"

This isn't exactly what you're looking for, but I've found this ranked list of llms useful -- https://huggingface.co/spaces/lmsys/chatbot-arena-leaderboar... Something more domain/use-specific would be great to have

Keep in mind that the rankings are just that: an order of "best" to "worst". There's nothing on there that's an indication of how much better one model is from another.

Re: Mistral-8x7B-Chat

#62

Post by kristianp »

kristianp wrote: post #4
antirez wrote: post #2

llama.cpp support needed ASAP, not just quantized, but I hope it will be albe to run in a way that keeps the same two experts for minimum N tokens generated and re-valuate the expert to select only then, so that it will be possible to do the inference in low ram doing something similar to "paging".

There is an issue for it: https://github.com/ggerganov/llama.cpp/issues/4381

I'm looking forward to a 4-bit quantized version that I can run on my Intel Thinkpad with 32GB RAM, should be almost usable if its as fast as a 14B size model.

Whether it's any good is another matter, I guess the leaderboard on HuggingFace will be updated at some point.

Re: Mistral-8x7B-Chat

#64

Post by WhitneyLand »

stavros wrote: post #61
gregsadetsky wrote: post #45

Earlier quoted context omitted.

This isn't exactly what you're looking for, but I've found this ranked list of llms useful -- https://huggingface.co/spaces/lmsys/chatbot-arena-leaderboar... Something more domain/use-specific would be great to have

Keep in mind that the rankings are just that: an order of "best" to "worst". There's nothing on there that's an indication of how much better one model is from another.

That’s exactly what it does.

It’s an Elo scoring system, like chess. By definition the difference between two entries represents how often it beats the other entry.

https://en.wikipedia.org/wiki/Elo_rating_system

Re: Mistral-8x7B-Chat

#65

Post by stavros »

WhitneyLand wrote: post #64
stavros wrote: post #61

Earlier quoted context omitted.

Keep in mind that the rankings are just that: an order of "best" to "worst". There's nothing on there that's an indication of how much better one model is from another.

That’s exactly what it does. It’s an Elo scoring system, like chess. By definition the difference between two entries represents how often it beats the other entry. https://en.wikipedia.org/wiki/Elo_rating_system

So, GPT-4 beats the next best LLM. Is GPT-4 twice as good, or a hundred times as good as the next one?

Re: Mistral-8x7B-Chat

#66

Post by Der_Einzige »

PostOnce wrote: post #57
behnamoh wrote: post #27

Earlier quoted context omitted.

> Is there a centralized list somewhere that tests "use this for x purpose, use that for y?" Yeah, "don't use these models for production, use OpenAI for production, ignore Claude/Gemini/etc.".

"Put all your eggs in one basket" isn't the traditional advice. Any one of these companies can at any time change their API, pricing, access rules, or even swap the model out for a dumber, cheaper one at the same price. You'll have no recourse if you don't have several backends available, or control your own. At a minimum, you should have available several hot-swappable backends/APIs if you want to remain viable in a…

I want someone to offer a large sum of money to anyone who can leak the full GPT-4 model to prevent ClosedAIs lunacy. We already got a model leak from NAI and it forever propelled AI waifus. Why can’t some Good Samaritan take one for the team and give the world access to the closest thing we have to AGI? Whoever does it will be remembered even more positively than Aaron Swartz

Why should I respect copyright on a model when the model training didn’t respect copyright? To be clear, I’m a copyright abolitionist so I’m down with not accepting copyright. I want someone to force OpenAI to be actually open.

Re: Mistral-8x7B-Chat

#67

Post by atcalan »

Jimbabwe wrote: post #26
curious_cat_163 wrote: post #14

Earlier quoted context omitted.

You'll want to try llama.cpp [1]. The set of models that it can support is expanding [2]. Folks have also written services [3] that wrap around it. [1] https://github.com/ggerganov/llama.cpp [2] https://huggingface.co/TheBloke [3] https://github.com/abetlen/llama-cpp-python

Thanks! I was just following the thread about their recent addition of the OpenCl support and was on the verge of trying it out last weekend. I’ll definitely continue once I’m home again!

It runs up to 13b models nicely on a 16g orange pi 5 fyi.

Re: Mistral-8x7B-Chat

#68

Post by atcalan »

brucethemoose2 wrote: post #48
holoduke wrote: post #40

Earlier quoted context omitted.

Save yourself some time and buy a 4090 if you really want to be high tier (consumer range) you will have a much faster experience. Not only with text. Also stable diffusion etc

Or better yet, a 3090 for very similar performance at half the price.

Why not both? Or two 3090s for that sweet, sweet nvlink for fine tuning?

Re: Mistral-8x7B-Chat

#69

Post by appplication »

lannisterstark wrote: post #5

Man this LLM stuff gets released faster than I can keep up. Is there a centralized list somewhere that tests "use this for x purpose, use that for y?"

Honestly none of them are particularly incredible compared to neither. At this point everything is a prototype, there’s no sense in building knowledge specific to them unless you’re also building LLM products.

It’s just distributed ADHD at this point. LLMs are new and cool and each new release will be significantly better than the last, but as with any emerging tech, we’re on an exponential curve so there’s no sense in falling in love with specifics or products until things stabilize.

Re: Mistral-8x7B-Chat

#70

Post by furyofantares »

stavros wrote: post #65
WhitneyLand wrote: post #64

Earlier quoted context omitted.

That’s exactly what it does. It’s an Elo scoring system, like chess. By definition the difference between two entries represents how often it beats the other entry. https://en.wikipedia.org/wiki/Elo_rating_system

So, GPT-4 beats the next best LLM. Is GPT-4 twice as good, or a hundred times as good as the next one?

Looks like 62 points better, which under the Elo model implies about 59/41 matchup (aka GPT-4-Turbo is about 59% chance to beat Claude on a random prompt). You can look up an Elo probability calculator to plug in an Elo difference and see this number. For example plug in two Elo ratings here: https://sandhoefner.github.io/chess.html - really only the difference between them matters.

Btw the similarity of the numbers 59 and 62 here is coincidence, other differences won't be nearly the same as the probability.

Post reply on HN