Viewing profile — johndough

HuggingFace used an NVFP4 quant of GLM-5.2 to investigate their latest hack, so that might also be worth a try: https://huggingface.co/nvidia/GLM-5.2-NVFP4

> Smaller models have less entropy. Interesting. Why is that? I would have expected the opposite, since larger models have to try less hard to fit the training data. Or maybe this …

> If wonder if you can train a model to optimize this, by trying to make the expert selection sticky across a few tokens You can! > AFM 3 Core Advanced makes routing decisions per …

Update: Looks like the model is larger after all (1561.44 GB). Only the MoE weights are MXFP4, while the other weights are BF16 (and a few FP32). * Sparse Experts: 1481.4 GB * Dens…

> But I think it's going to need more than 1536GB of RAM, with a usable and large amount of context, more like 2TB and preferably 2.5 to 3TB. The model is known to be MXFP4 accordi…

DeepSeek-V4 should use only 5GB for context due to CSA and HCA, see figure here: https://huggingface.co/deepseek-ai/DeepSeek-V4-Pro But not every framework implements it properly y…

> There is a larger 120B that you can't realistically run on consumer hardware at reasonable tok/s too. gpt-oss-120b runs at 30+ tps on Strix Halo and +75 tps on a MacBook Pro M5 M…

Why not? The model is 2.8T parameters with native MXFP4, which is 1400GB.

> Once the tide drops enough for hardware lead times to shorten to weeks Which will not be any time soon according to SK Hynix CEO: > We still forecast that customer demand will re…

> GPUs become obsolete in 5 years The GPUs are far from worthless after 5 years. E.g. the A100 80GB PCIe version cost around $15k when it was introduced in 2021 and now sells for $…

Thanks for pointing that out. I haven't noticed any adds in years with Firefox and Ublock Origin extension. I'll look for a better image host in the future. I guess the economic in…

Oops, autocorrect. Sorry about that.

Another point for consideration: Specialized SVG models create way better looking pelicans riding a bicycle. (E.g. Refract V4: https://jumpshare.com/s/8liB7Aiuoo3yucbWGXjZ mirror: …

I would not put any trust into that AI-generated "research". It does not control for how OpenAI, Anthropic and Moonshot are doing tokenization and token healing, so the results are…

Neat, I did not know of that functionality. Thanks!

I am not arguing that Kimi did not distill Opus or Sonnet (which they almost certainly have, as opposed to Fable or Sol). I am arguing that even if they did not distill, the model …

Implementing your own agent is very easy. Here is a minimal agent in 60 lines of Python without dependencies: https://github.com/99991/MinimalAgent You only need a single tool to s…

The exact model identifiers appear extremely frequently in code on GitHub. https://grep.app/search?q=claude-opus-4-5-20251101 https://grep.app/search?q=claude-sonnet-4-20250514 The…

> Zero emissions, silent, no illumination, etc. Mostly silent. High voltage transmission lines may have a low-frequency hum. You have to stand directly under them to hear it though…

But then again, the identity could also have slipped into the model from other sources during pretraining. The internet is full of "I am Claude": https://grep.app/search?q=i+am+cla…

Your comment sounds like an AI-generated advertisement.

If an uneducated acquaintance of yours was about to name their newborn child "Adolf", would you interject that this name might not be such a great idea?

It would be great if this could be combined with quantization-aware finetuning. In my experience, Qwen3.6-27B has much fewer repetitions at Q6 quantization level as compared to e.g…