Those rope tests are impressive AF
Qwen3-Next
161–170 of 240 posts
Re: Qwen3-Next
#162llm -m qwen3-next-80b-a3b-thinking "An ASCII of spongebob" Here's a classic ASCII art representation of SpongeBob SquarePants: .------. / o o \ | | | \___/ | \_______/ llm -m chutes/Qwen/Qwen3-Coder-480B-A35B-Instruct-FP8 \ "An ASCII of spongebob" Here's an ASCII art of SpongeBob SquarePants: .--..--..--..--..--..--. .' \ (`._ (_) _ \ .' | '._) (_) | \ _.')\ .----..--.' / |(_.' | / .-\-. \---. \ 0| | ( O| O) | | | _…
[deleted]
Re: Qwen3-Next
#163ICYMI qwen3-max was released last week.
Was Qwen3-max better than Qwen3-235B-A22B-2507 at anything? Except higher token limit?
Re: Qwen3-Next
#164llm -m qwen3-next-80b-a3b-thinking "An ASCII of spongebob" Here's a classic ASCII art representation of SpongeBob SquarePants: .------. / o o \ | | | \___/ | \_______/ llm -m chutes/Qwen/Qwen3-Coder-480B-A35B-Instruct-FP8 \ "An ASCII of spongebob" Here's an ASCII art of SpongeBob SquarePants: .--..--..--..--..--..--. .' \ (`._ (_) _ \ .' | '._) (_) | \ _.')\ .----..--.' / |(_.' | / .-\-. \---. \ 0| | ( O| O) | | | _…
I realize my SpongeBob post came off flippant, and that wasn't the intent. The Spongebob ASCII test (picked up from Qwen's own Twitter) is explicitly a rote-memorization probe; bigger dense models usually ace it because sheer parameter count can store the sequence
With Qwen3's sparse-MoE, though, the path to that memory is noisier: two extra stochastic draws (a) which expert(s) fire, (b) which token gets sampled from them. Add the new gated-attention and multi-token heads and you've got a pipeline where a single routing flake or a dud expert can break vertical alignment halfway down the picture.
Anyway, I think qwen3-coder was uniquely trained on this - so it's not a fair comparison. Here are some other qwen3 models:
Model: chutes/Qwen/Qwen3-235B-A22B
/~\
( * * )
( o o o )
\ - /
\ /\ /
\ /
\/
/|||\
/|||||\
/||||||||\
( o o o )
\ W /
\___/
/\_/\
( o.o )
> ^ .-----------.
| [] [] |
| |
| __ __ |
| | || | |
| |__||__| |
| |
'-----------'
__
/ \
/ \
/ \
/ \
/__________\
| o o |
| __ |
| \____/ |
| |
| ________ |
| \ / |
| \____/ |
| |
|____________|
___________
/ \
/ \
| _______ |
| | | |
| | ___ | |
| | | | | |
| | |___| | |
| |_______| |
| |
|_______________|
.-.
/ \
| o o|
| > |
| ---|
\___/
_________________________
/ \
| _ _ _ _ |
| / \ / \ / \ / \ |
| | | | | | | | | |
| \_/ \_/ \_/ \_/ |
| |
| _ _ _ _ |
| / \ / \ / \ / \ |
| | | | | | | | | |
| \_/ \_/ \_/ \_/ |
| |
| SpongeBob SquarePants |
|_________________________|Re: Qwen3-Next
#165Earlier quoted context omitted.
Could someone kindly point to a convenient all-on-one ELI5 of all these words? :')
Background: LLMs take your input, upscale it into a very high dimensional space, and then downscale it back to 1D at the end. This 1D list is interpreted as a list of probabilities -- one for each word in your vocabulary. i.e f(x) = downscale(upscale(x)). Each of downscale() and upscale() are parameterized (billions of params). I see you have a gamedev background, so as an example: bezier curves are parameterized fun…
Ooooh, neat! That was very well explained, thank you.
Re: Qwen3-Next
#166Hmm. 80B. These days I am on the lookout for new models in the 32B range, since that is what fits and runs comfortably on my MacBook Pro (M4, 64GB). I use ollama every day for spam filtering: gemma3:27b works great, but I use gpt-oss:20b on a daily basis because it's so much faster and comparable in performance.
it'll run great, it's an moe.
[deleted]
Re: Qwen3-Next
#167Earlier quoted context omitted.
Isn't that essentially how the MoE models already work? Besides, if that were infinitely scalable, wouldn't we have a subset of super-smart models already at very high cost? Besides, this would only apply for very few use cases. For a lot of basic customer care work, programming, quick research, I would say LLMs are already quite good without running it 100X.
> if that were infinitely scalable, wouldn't we have a subset of super-smart models already at very high cost The compute/intelligence curve is not a straight line. It's probably more a curve that saturates, at like 70% of human intelligence. More compute still means more intelligence. But you'll never reach 100% human intelligence. It saturates way below that.
how would you know it converges on human limits, why wouldn't it be able to go beyond, especially if it gets its own world sim sandbox?
Re: Qwen3-Next
#168All these new datacenters are going to be a huge sunk cost. Why would you pay OpenAI when you can host your own hyper efficient Chinese model for like 90% less cost at 90% of the performance. At that is compared to today's subsidized pricing, which they can't keep up forever.
>to today's subsidized pricing, which they can't keep up forever.
The APIs are not subsidized, they probably have quite the large margin actually: https://lmsys.org/blog/2025-05-05-large-scale-ep/
>Why would you pay OpenAI when you can host your own hyper efficient Chinese model
The 48GB of VRAM or unified memory required to run this model at 4bits is not free either.
Re: Qwen3-Next
#169Earlier quoted context omitted.
Yes - they all do that. Actually, most attempts start well but unravel toward the end. llm -m chutes/Qwen/Qwen3-Coder-480B-A35B-Instruct-FP8 \ "An ASCII of spongebob" Here's an ASCII art of SpongeBob SquarePants: ``` .--..--..--..--..--..--. .' \ (`._ (_) _ \ .' | '._) (_) | \ _.')\ .----..--. / |(_.' | / .-\-. \ \ 0| | ( O| O) | | _ | .--.____.'._.-. /.' ) | (_.' .-'"`-. _.-._.-.--.-. / .''. | .' `-. .-'-. .-'"`-.`-…
Ph'nglui mglw'nafh Cthulhu R'lyeh wgah'nagl fhtagn.
Yes, Mylord! We'll find and destroy all of that damn shrubbery!
Re: Qwen3-Next
#170All these new datacenters are going to be a huge sunk cost. Why would you pay OpenAI when you can host your own hyper efficient Chinese model for like 90% less cost at 90% of the performance. At that is compared to today's subsidized pricing, which they can't keep up forever.
Eventually Nvidia or a shrewd competitor will release 64/128gb consumer cards; locally hosted GPT 3.5+ is right around the corner, we're just waiting for consumer hardware to catch up at this point.