What is a "credit" and how does it translate to tokens for the different models?
Its the currency of the future.
They use it in Babylon5 (supposedly) in 2260!
Qwen 3.8
211–220 of 793 posts
Re: Qwen 3.8
#212Earlier quoted context omitted.
> Qwen 3.7 Pro is just unusable. wastes too much time, goes off track, useless stuck loops, cannot debug at all. D Anthropic should not have bugged their knowledge distillation attacks.
> Anthropic should not have bugged their knowledge distillation attacks. It is like one of Pizzaro's men crying that someone have stolen his precious golden dublons As Lenin have said - "Loot the looters" (Russian: Грабь награбленное)
Appealing to the Belsheviks for moral authority is, well I will just say an interesting approach. I do not have that much sympathy for Anthropic, but I do not have much sympathy for publishing companies either whose rights to a revenue stream they violated either. Are Chinese AI companies the Robin Hood in this story? Would they be so magnanimous if they had the upper hand? I don’t think so.
Re: Qwen 3.8
#213Does anyone have the privacy policy of their token plan available? Want to check if they retain/train on inputs/outputs.
Lol, lmao even. Of course they train on literally everything they get their hands on, like everyone else. If you need privacy, that's what local models are for.
It's a flippant answer to a real question. Anthropic, OpenAI, and even Grok have "Don't train on my data" knobs.
Whether you trust them is different, but there ARE knobs on other hosted AI companies.
Re: Qwen 3.8
#214Re: Qwen 3.8
#215Earlier quoted context omitted.
Everyone wanted open models that would challenge Opus and Codex, here, you got it.
We need better coding models that can run on local hardware, i.e. 128GB VRAM or less
You can run larger models by offloading to SSD (for weights), it's just slow so people don't do it all that much. But you can get back at least some of that performance by using either MTP (at least for dense models; not effective for sparse MoE models unless you're batching them already and have VASTLY more parallel compute than you'd know what to do with) or batching multiple requests in parallel (note, this hurts throughput for your single sessions but running more sessions in parallel still boosts your total amount of inference. This requires careful management of memory requirements for your context/KV cache, and Qwen models tend to be KV-cache heavy).
Broadly speaking, this ultimately pushes local inference towards a challenging world where you use SSD offload for weights as a matter of course; then smaller requests (or requests sharing the bulk of their context, e.g. subagent swarms) can be batched together and run quickly in aggregate, but running very large contexts will actually limit you to single-session inference and require swapping out even the KV cache itself to some external scratch SSD, further hurting your performance. Then feel free to add wide use of MTP in a probably futile effort to go back to tolerable tok/s numbers.
Re: Qwen 3.8
#216[dead]
Re: Qwen 3.8
#217Earlier quoted context omitted.
It's hard to say what their motivation is. The Chinese firms seem to be working hard to commoditize intelligence which may be the most effective way to debase American frontier labs. And yeah: it also happens to be really good for humanity.
China is watching world sentiment shifting away from USA. Doing many small things that show both strength and openness is surely very intentional.
The US leadership (both government and industry) really seems set on making everyone go with the Chinese competition at this point.
Re: Qwen 3.8
#218Earlier quoted context omitted.
I know this is a bit cliche but I wonder how much headroom there is in the lower parameter count range. Is there any good reason to believe there is a lot of headroom or there is not? I suppose I'm just wondering if this wave of nearly Fable class models will be runnable on ~$10k worth of hardware at reasonable speeds in the near future.
> I suppose I'm just wondering if this wave of nearly Fable class models will be runnable on ~$10k worth of hardware at reasonable speeds in the near future You're able to run quantized ~100B class models on local hardware today, but still lots of compromises when it comes to quality. I guess it ultimately depends on how far "near future" is, in a year you'd likely be able to run something like 5.6 Terra on local (~1…
I think it’s mainly a question of whether the price-fixing of VRAM continues or whether an inflection point is forced by the low margins of the industry and potential supply increases. Once the normal scaling of hardware and prices resumes, it’s game over for proprietary, which is why there’s so much urgency to seek market control instead right now.
Re: Qwen 3.8
#219SVG's pelican https://gist.github.com/vitordelucca/521c2d63c9b852c622e7648... Made on the website, so not sure if on the API there's more thinking options...
I feel like the pelican test can't be relevant anymore; the whole point was to to something that wouldn't be in the training set at all and now it is?
A parachuting flamingo? An aardvark driving a bus? It should be easy to randomize the animal and the mode of transport (or vary it with animal playing a sport) to create images not in the training data.
Re: Qwen 3.8
#220I predict that no one will use this and everyone will use Kimi K3.
We'll probably use it, but for images. Qwen is still the best for images