Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Mistral-8x7B-Chat

huggingface.co

Post reply on HN
Print view View original

51–60 of 75 posts

Re: Mistral-8x7B-Chat

#51

Post by brucethemoose2 »

refulgentis wrote: post #24
brucethemoose2 wrote: post #12

Earlier quoted context omitted.

> qnap tvs-473e Specs say this runs an AMD RX-421BD. This is a 2015 AMD CPU with 2 bulldozer cores and a tiny IGP. ...To be blunt, you would be much better off running LLMs on your phone. Even an older phone. Or literally whatever device you are reading HN on. But if you insist , the runtime you want in MLC-LLM's Vulkan runtime.

This. Sibling llama.cpp comment is standard "I know llama.cpp, I assume that's 80% of the universe instead of .8%, and I assume that's all anyone needs. So I know just enough to be dangerous with ppl looking for advice". You'll see it over and over again when you're looking for help, be careful, it's 100% a blind alley in your case. It's very likely you'll be disappointed by MLC as well, simultaneously it's your only…

Yeah I feel this way about llama.cpp too. Maybe not that exact wording, but its not the end all be all llama runtime like everyone seems to think it is.

Re: Mistral-8x7B-Chat

#52

Post by [deleted] »

tempaway47641 wrote: post #41

Every day: "Cheeseface just dropped the Blippy-7B model which is almost as good as the twinamp 34B model on the SwagCube benchmark when run locally as int8 and this shows that the gains made by the skibidi-70B model will probably filter down to the baseline Eras models in the next few weeks"

[deleted]

Re: Mistral-8x7B-Chat

#53

Post by kostadin »

jaipilot747 wrote: post #20
rgbrgb wrote: post #9

Earlier quoted context omitted.

what a sick project to be able to attract a billionaire programmer [0] and c royalty. [0]: https://github.com/ggerganov/llama.cpp/issues/4216#issuecomm...

For those out of the loop, who are the billionaire programmer and C royalty people in this link?

Tobi Shopify and antirez (redis etc)

Re: Mistral-8x7B-Chat

#54

Post by king_magic »

a_wild_dandan wrote: post #3

PSA: This is broken for Apple silicon. It relies on the CUDA-only `flash_attn` package. Alas, I wanted to try Mistral's MoE on my MacBook. :( Understandable though! It's a new model with an incredibly bizarre release. My phalanges are crossed.

I think "broken" is a bit of an exaggeration, when the actual problem is "just not compatible with your hardware".

Re: Mistral-8x7B-Chat

#56

Post by htrp »

brucethemoose2 wrote: post #7
lannisterstark wrote: post #5

Man this LLM stuff gets released faster than I can keep up. Is there a centralized list somewhere that tests "use this for x purpose, use that for y?"

Even the OSS trainers/finetuners themselves don't really know the use case for their models. They just hot drop them and move on, lol.

They train and drop them.... and rely on you to figure it out.

Re: Mistral-8x7B-Chat

#57

Post by PostOnce »

behnamoh wrote: post #27
lannisterstark wrote: post #5

Man this LLM stuff gets released faster than I can keep up. Is there a centralized list somewhere that tests "use this for x purpose, use that for y?"

> Is there a centralized list somewhere that tests "use this for x purpose, use that for y?" Yeah, "don't use these models for production, use OpenAI for production, ignore Claude/Gemini/etc.".

"Put all your eggs in one basket" isn't the traditional advice.

Any one of these companies can at any time change their API, pricing, access rules, or even swap the model out for a dumber, cheaper one at the same price. You'll have no recourse if you don't have several backends available, or control your own.

At a minimum, you should have available several hot-swappable backends/APIs if you want to remain viable in an indeterminate future.

Re: Mistral-8x7B-Chat

#58

Post by OJFord »

jaipilot747 wrote: post #20
rgbrgb wrote: post #9

Earlier quoted context omitted.

what a sick project to be able to attract a billionaire programmer [0] and c royalty. [0]: https://github.com/ggerganov/llama.cpp/issues/4216#issuecomm...

For those out of the loop, who are the billionaire programmer and C royalty people in this link?

The linked comment is authored by the CEO of Shopify; the top-level comment in this thread is the .. what would you say, author/originator/founder of Redis. (And maybe/presumably more widely known in C-land for other projects or contributions, idk.)

Re: Mistral-8x7B-Chat

#59

Post by sebosp »

brucethemoose2 wrote: post #47
tempaway47641 wrote: post #41

Every day: "Cheeseface just dropped the Blippy-7B model which is almost as good as the twinamp 34B model on the SwagCube benchmark when run locally as int8 and this shows that the gains made by the skibidi-70B model will probably filter down to the baseline Eras models in the next few weeks"

Thats giving the community way too much organization credit. Everyone just seems to be running experiments independently and then randomly drop some results, with basically no documentation. Sometimes the motivation is clearly VC money or paper exposure, but sometimes there is no apparent motivation... Or even no model card. Then when something works, others copy the script. Not that I dont enjoy it. I find the sea o…

Now I feel I have to read again Haroun and the Sea of Stories from Rushdie thanks to your last comment <3 that's one of my favorite childhood reads
Post reply on HN