31–40 of 75 posts
We just made it available on OpenRouter: https://openrouter.ai/models/fireworks/mixtral-8x7b-fw-chat Chat playground: https://openrouter.ai/playground?models=fireworks/mixtral-8x...
There’s probably a better place to ask this highly specific technical question, but I’m avoiding Reddit these days so just throwing it out I guess. I’ve been trying to run these in a container but it’s verrrry slow, I believe, because of the lack of gpu help. All the instructions I find are for nvidia gpus and my server is a qnap tvs-473e with an embedded amd cpu/gpu (I know, I know). The only good news is that I’ve…
> my server is a qnap tvs-473e with an embedded amd cpu/gpu That's your problem. I googled and it looks like one of these all-in-one appliances like a drobo or whatever's popular these days. That's not a server. (At least, I wouldn't call it a server. It's an all-in-one appliance, or toy, depending on perspective) And yegods, that price... Spend $500, get an actual computer, not some priced up appliance, and you'll h…
Earlier quoted context omitted.
> qnap tvs-473e Specs say this runs an AMD RX-421BD. This is a 2015 AMD CPU with 2 bulldozer cores and a tiny IGP. ...To be blunt, you would be much better off running LLMs on your phone. Even an older phone. Or literally whatever device you are reading HN on. But if you insist , the runtime you want in MLC-LLM's Vulkan runtime.
This. Sibling llama.cpp comment is standard "I know llama.cpp, I assume that's 80% of the universe instead of .8%, and I assume that's all anyone needs. So I know just enough to be dangerous with ppl looking for advice". You'll see it over and over again when you're looking for help, be careful, it's 100% a blind alley in your case. It's very likely you'll be disappointed by MLC as well, simultaneously it's your only…
Sorry for reddit link.
This model is better by many other contenders, but still far from GPT4. "what famous brands are there which change one letter from a common word to make a non-existent, but a catchy name, such as "musiq" instead of "music".. etc?" There are several brands that have played with words by changing a letter or adding a letter to create a new and memorable name. Here are a few examples: Qatar Airways - This airline's name…
But both are completely wrong! And technically the Google example is closer to correct than any others. The Yi 34B eBay and Kodak examples are both (wrong but) very interesting because it does seem to get the idea of changing one letter. Of GPT4 examples, the Qatar example (replacing "Q" with "Q" !?) is the only one that is internally consistent. The Pinterest and Tumblr examples are wrong in very odd ways in that th…
That appears to be from vizzah's testing of Mistral-8x7B-Chat rather than GPT4.
Earlier quoted context omitted.
This. Sibling llama.cpp comment is standard "I know llama.cpp, I assume that's 80% of the universe instead of .8%, and I assume that's all anyone needs. So I know just enough to be dangerous with ppl looking for advice". You'll see it over and over again when you're looking for help, be careful, it's 100% a blind alley in your case. It's very likely you'll be disappointed by MLC as well, simultaneously it's your only…
Thanks. Yeah I’ve noticed that llama.cpp is definitely the recommendation du jour, but this thread gave me hope: https://www.reddit.com/r/LocalLLaMA/comments/13p8zq2/update_... Sorry for reddit link.
To be blunt, there isn't much interest in support outside of apple/nvidia. There is a WIP Vulkan backend, but (last I checked) progress is slow and its not optimzed for IGPs either.
MLC-LLM is much more promising once its features get fleshed out, as it "inherits" support for many devices from its Apache TVM backend.
There’s probably a better place to ask this highly specific technical question, but I’m avoiding Reddit these days so just throwing it out I guess. I’ve been trying to run these in a container but it’s verrrry slow, I believe, because of the lack of gpu help. All the instructions I find are for nvidia gpus and my server is a qnap tvs-473e with an embedded amd cpu/gpu (I know, I know). The only good news is that I’ve…
> my server is a qnap tvs-473e with an embedded amd cpu/gpu That's your problem. I googled and it looks like one of these all-in-one appliances like a drobo or whatever's popular these days. That's not a server. (At least, I wouldn't call it a server. It's an all-in-one appliance, or toy, depending on perspective) And yegods, that price... Spend $500, get an actual computer, not some priced up appliance, and you'll h…
Earlier quoted context omitted.
> my server is a qnap tvs-473e with an embedded amd cpu/gpu That's your problem. I googled and it looks like one of these all-in-one appliances like a drobo or whatever's popular these days. That's not a server. (At least, I wouldn't call it a server. It's an all-in-one appliance, or toy, depending on perspective) And yegods, that price... Spend $500, get an actual computer, not some priced up appliance, and you'll h…
Something to add: if your NAS-Server is advertised with it's dual 4K capable HDMI ports, you know you're looking at a consumer toy targeted at people who hoard (pirated) media and not for serious stuff.
I hadn't even noticed that, I just saw "I could build that for 1/4th the cost"+"wtf, only 4 drives?"
Stuff like this prompts a dual response in me. :)
It always gives me a strong urge to educate "it's not that hard, and it's fun" to build it yourself.
AND it always makes me kick myself for not commercializing the expandable media servers I started building in the early 2000s, for me, for the dorm, for my friends, i.e. exactly the people you identify. :)
Man this LLM stuff gets released faster than I can keep up. Is there a centralized list somewhere that tests "use this for x purpose, use that for y?"
> Is there a centralized list somewhere that tests "use this for x purpose, use that for y?" Yeah, "don't use these models for production, use OpenAI for production, ignore Claude/Gemini/etc.".
I thought the new MoE Mistral model was called Mixtral.
How long till llm-mixtral?
Btw. The other day I uploaded ~4k chats from your llm dB to the code interpreter and had it label them. Worked pretty well. Only gave them one label each at first, then I started working on more complex labeling, but then my poor assistant ran out of steam and the interpreter session expired. So much pleasure and pain! Love your llm cli though. Thanks.
There’s probably a better place to ask this highly specific technical question, but I’m avoiding Reddit these days so just throwing it out I guess. I’ve been trying to run these in a container but it’s verrrry slow, I believe, because of the lack of gpu help. All the instructions I find are for nvidia gpus and my server is a qnap tvs-473e with an embedded amd cpu/gpu (I know, I know). The only good news is that I’ve…