Live data from Hacker News

Mistral "Mixtral" 8x7B 32k model [magnet]

twitter.com

251–255 of 255 posts

Re: Mistral "Mixtral" 8x7B 32k model [magnet]

#251

Earlier quoted context omitted.

It's much easier to do RAG than try to shoehorn the entirety of the universe into 7B parameters every 24 hours. Mistral's great at being coherent and processing info at 7B, but you wouldn't want it as an oracle.

I didn't know about RAG, thanks for sharing. I am not sure, if outdated information can be tackled with RAG though, especially in coding. Just today, i asked GPT and Bard(Gemini) to write code using slint, neither of them had any idea of slint. Slint being a relatively new library, like two and a half (0.1 version) to one and a half (0.2 version) years back [1] is not something they trained on. Natural language doesn…

Did you use gpt4? I asked it to give me some examples code for a dialog box with an input field and a button and it gave me this:

use slint::slint;

slint! { DialogBox := Window { width: 400px; height: 200px; title: "Input Dialog";

        VerticalBox {
            padding: 20px;
            spacing: 10px;

            TextInput {
                id: input_field;
                placeholder_text: "Enter text here";
            }

            Button {
                text: "Submit";
                clicked => {
                    // Handle the button click event
                    println!("Input: {}", input_field.text());
                }
            }
        }
    }
}

fn main() { DialogBox::new().run(); }

Re: Mistral "Mixtral" 8x7B 32k model [magnet]

#252

Earlier quoted context omitted.

I just played with 7b version. It really feels different than anything I tried before. It could explain a docker compose file. It generated a simple vue application component. I asked around a bit about the example and it was strangely coherent and focused across the whole conversation. It was really well detecting, where I'm starting a new thread (without clearing a context) or referring to things before. It caught…

How did you run it? Are there model files in Ollama format? Are you running on NVidia or Apple Silicon? EDIT: just saw this “ Megatron (1, 2, and 3) is a large, powerful transformer developed by the Applied Deep Learning Research team at NVIDIA.”

your edit is entirely unrelated to this topic

Re: Mistral "Mixtral" 8x7B 32k model [magnet]

#253
post #145

In other llm news, Mistral/Yi finetunes trained with a new (still undocumented) technique called "neural alignment" are blasting other models in the HF leaderboard. The 7B is "beating" most 70Bs. The 34B in testing seems... Very good: https://huggingface.co/fblgit/una-xaberius-34b-v1beta https://huggingface.co/fblgit/una-cybertron-7b-v2-bf16 I mention this because it could theoretically be applied to Mistral Moe. If…

Correct. UNA can align the MoE at multiple layers, experts, nearly any part of the neural network I would say. Xaberius 34B v1 "BETA".. is the king, and its just that.. the beta. I'll be focusing on the Mixtral, its a christmas gift.. modular in that way, thanks for the lab @mistral!

Do you have any docs describing the method?

Re: Mistral "Mixtral" 8x7B 32k model [magnet]

#254

Earlier quoted context omitted.

I didn't know about RAG, thanks for sharing. I am not sure, if outdated information can be tackled with RAG though, especially in coding. Just today, i asked GPT and Bard(Gemini) to write code using slint, neither of them had any idea of slint. Slint being a relatively new library, like two and a half (0.1 version) to one and a half (0.2 version) years back [1] is not something they trained on. Natural language doesn…

Did you use gpt4? I asked it to give me some examples code for a dialog box with an input field and a button and it gave me this: use slint::slint; slint! { DialogBox := Window { width: 400px; height: 200px; title: "Input Dialog"; VerticalBox { padding: 20px; spacing: 10px; TextInput { id: input_field; placeholder_text: "Enter text here"; } Button { text: "Submit"; clicked => { // Handle the button click event printl…

Thanks for letting me know, i didn't use GPT-4, but i was under the impression that the cutoff data between all GPT's was the same, or almost the same. The code is correct, yes.

I do not have a GPT4 subscription, i did not bother because it is so slow, limited queries etc. If the cutoff date is improved, like being updated periodically i may think about it. (Late response, forgot about the comment!)

Re: Mistral "Mixtral" 8x7B 32k model [magnet]

#255

Earlier quoted context omitted.

Did you use gpt4? I asked it to give me some examples code for a dialog box with an input field and a button and it gave me this: use slint::slint; slint! { DialogBox := Window { width: 400px; height: 200px; title: "Input Dialog"; VerticalBox { padding: 20px; spacing: 10px; TextInput { id: input_field; placeholder_text: "Enter text here"; } Button { text: "Submit"; clicked => { // Handle the button click event printl…

Thanks for letting me know, i didn't use GPT-4, but i was under the impression that the cutoff data between all GPT's was the same, or almost the same. The code is correct, yes. I do not have a GPT4 subscription, i did not bother because it is so slow, limited queries etc. If the cutoff date is improved, like being updated periodically i may think about it. (Late response, forgot about the comment!)

Yes it’s much better now in all those areas, I think you’ll be surprised if your last experience was a few months ago. The difference in ability between 3.5 and 4 is significant.
Post reply on HN