DFlash 2: Keep Drafting Parallel
1–10 of 20 posts
Re: DFlash 2: Keep Drafting Parallel
#2vllm PR for DFlash2: https://github.com/vllm-project/vllm/pull/52816
Re: DFlash 2: Keep Drafting Parallel
#3I'm getting around 27 tokens per second decode using vLLM + Qwen 3.8 27b nvfp4 + DFlash 2 on the DGX Spark.
Re: DFlash 2: Keep Drafting Parallel
#4Amazing tech
> An agent writes in an afternoon what a chatbot writes in a month
But can you just.. not.
Your tech is so good, it speaks for itself. Don't ruin that.
Re: DFlash 2: Keep Drafting Parallel
#5Great news, has made low memory bandwidth model usage so much nicer.
Re: DFlash 2: Keep Drafting Parallel
#6vllm PR for DFlash2: https://github.com/vllm-project/vllm/pull/52816
llama.cpp PR https://github.com/ggml-org/llama.cpp/pull/27342
Re: DFlash 2: Keep Drafting Parallel
#7Watch the video carefully. DFlash2's tool call fails on python syntax.
Usually models in this class nail things like that 1 shot, which the other side did.
I don't know the cause. It may be nothing. But I'd like to see the model doing something where its path is a bit more constrained, to help out rule out such oddities.
Re: DFlash 2: Keep Drafting Parallel
#8Watch the video carefully. DFlash2's tool call fails on python syntax. Usually models in this class nail things like that 1 shot, which the other side did. I don't know the cause. It may be nothing. But I'd like to see the model doing something where its path is a bit more constrained, to help out rule out such oddities.
DFlash is lossless so this would be a bug in the implementation if it is indeed a regression against the target model.
Re: DFlash 2: Keep Drafting Parallel
#9Watch the video carefully. DFlash2's tool call fails on python syntax. Usually models in this class nail things like that 1 shot, which the other side did. I don't know the cause. It may be nothing. But I'd like to see the model doing something where its path is a bit more constrained, to help out rule out such oddities.
DFlash is lossless so this would be a bug in the implementation if it is indeed a regression against the target model.
Only if you do greedy sampling. With probabilisitic sampling (categorical sampling), you will end up with different trajectory just “mathematically equivalent”.
Re: DFlash 2: Keep Drafting Parallel
#10Earlier quoted context omitted.
DFlash is lossless so this would be a bug in the implementation if it is indeed a regression against the target model.
Only if you do greedy sampling. With probabilisitic sampling (categorical sampling), you will end up with different trajectory just “mathematically equivalent”.
Can you explain a little bit more please?