Viewing profile — liuliu

They ship a complete checkpoint for easily management (inference & training) in their own infrastructure. Moving to a LUT would make training on these layers impossible. BTW, these…

One thing similar would be projecting both the head.weight and the final LLM activations into a smaller vector space, since that is basically just cosine similarity ranking step (s…

One thing is not obvious to me is how ConvRot can be applicable beyond diffusion models. Especially for LLM decoding, as each ConvRot would be more expensive for a given decoding v…

It is a well-known trick, given that the timestep is between 0 to 1, you can slicing them at any resolution (1000, or 10000, give or take), and then keep a look-up table for modula…

Well, it is a "platform of Balancing Budget", you don't need to actually work on that or do anything. The harder part is just letting people believing in miracles. Look no further …

DS4 is designed to do real-work. Gemma 4 is not going to cut it.

> training an LLM takes more resources and expertise than distilling from an existing LLM This is not automatically true. Training and distillation use the same underlying infra an…

It is disabled because it doesn't work :) Try it and see the doom loop it gets itself in.

Note that 3.5 9B cannot do thinking (while 3.6 27B can, pretty effectively, quite verbosely).

You also need to pay close attention to BFCLv3 multi-turn result, that helps you to get a sense how frequently these quants will be in a doom loop.

The problem, of course, is if you run the UD_Q2 variant (Unsloth) which does only post-training, the number is pretty close to 1-bit model here and the 5% drop in tool-call is sign…

This should be the first comment here. Too many replies here are done before reading it. It is not "just another agent does the agent thing". It is a deliberate choice of the Grok …

Realistically, you need to experiment with any user prompt + a good amount of system prompt (at least > 1000 tokens, but realistically, in the range of 3000 tokens probably good). …

It is a “research note”. It might not pan out, and you might say it doesn’t deserve the attention on the internet. But it did suggest something that resembles of compression, just …

I actually agree. At some point, a RSI system has to interact with real-world, and that imposes serialization constraints. It is harder to know how much that slow-down would be and…

Except the two (GPT-Image-2 and Nano Banana Pro), anything displayed here can run on the 16 GiB MacBook (including the FLUX.2 [dev]): https://tests.drawthings.ai/generate

> To our knowledge, Bonsai Image 4B is the first image model in its parameter class to run directly on an iPhone. This is wrong. But they worded it carefully to be not entirely wro…

It solves part of the download issue if they actually delivers a 1-bit whole package (currently their download is around 3.5GiB, still not ideal since FLUX.2 [klein] 4B you can get…

One thing people seems not to acknowledge, and this post made it super clear is that NVIDIA kept their lead extremely well in a few years of very high growth. The TFLOPs, the bandw…

Probably not really. For gaming, I think probably just need to have a better way to explain visual and what the problem is (collision not done correctly, ways to feedback to LLM's …

Only if you think B is an important thing. He is easily > $100M from Tesla.

Since the frontier is only 8-month ahead of DeepSeek, it is hard to see how model training can be a moat as all the tricks are available from open labs in China. You really just ne…

DSv4 generates much faster on NVIDIA class hardware. It is just a very efficient model.

I am not sure where this comment is from (possibly without looking at this project?). This project is running quasi-frontier model at reasonable tps (~30) with reasonable prefill p…

Thanks. I think it is a good explanation, but also suggests a gap. QAT to me, if done right, is the only way to recover performance for extreme quantization regime. The only thing …