Viewing profile — woctordho

MiniMax H3 is going to release weights. You can locally run it with definitely less than $10k (and possibly faster than Seedance's queue), and it's fun to train it for whatever you…

GGUF is at least better than bnb. From what I know, bnb does not yet find a way to quantize MoE with enough accuracy, and maintain the dequant-MoE kernels. In the age of Qwen 3.0, …

Speaking of finetune, currently a common practice is LoRA over bnb 4-bit base model, but I think it's time to replace bnb with GGUF as the base model format. GGUF is actively suppo…

Distributed training is much harder than distributed inference but not impossible. See the recent development of DiLoCo at Nous Research and Prime Intellect.

Relevant: Why Switzerland has 25 Gbit internet and America doesn't https://news.ycombinator.com/item?id=47652400

AI Horde has some measures to prevent Sybil attack that returns wrong results, but not enforce zero data retention. Prompts belong to the whole open source community. For example h…

Petals is from 2022. Nowadays intelligence of smaller models, quantization techs, and optimizations to run models faster on consumer GPUs have improved a lot. For distributed infer…

So is making a PR different from making the whole software. This is what an open source community is good for.

See the recent development of DiLoCo at Nous Research and Prime Intellect.

There's a lot of individual effort of improving the models. See how many finetuned models and LoRAs are there on Hugging Face.

There is a forum named Zhihu. AI translation works mostly well to translate contents there into English.

Yes in a mid-sized company. I'm exactly doing this, and what I'm competing against is the OpenAI API priced 0.2 CNY = 1 USD in China.

There's nothing wrong to run CUDA on non-Nvidia hardware. CUDA has an interface that is reasonably well-designed, well-documented/reverse-engineered, and battle-tested for decades.…

And humans don't run on markets.

Fun fact: Hacker News is canonically banned in China, but I'm still talking here. There are plenty of techs to work around region block. The incentive to report somebody is comical…

See the recent advance of DiLoCo at Nous Research and Prime Intellect.

Don't trust US or China. Trust the open source community.

There are lots of botnets providing home IPs.

Lots of people have succeeded. Neither Anthropic nor OpenAI has any technical advantage in the field of subscription engineering.

Actually nowadays LLMs are only trained with TBs rather than PBs of data, and it's not too hard to find GBs of agent traces online.

Simple trick: Use an agentic tool like Pi or OpenCode that allows you to switch models. First do some chats with DeepSeek or GLM who shows full thinking traces, then switch to Clau…

There is already a lot of effort to collect agent traces including reasonings, e.g. see the recent discussion: https://old.reddit.com/r/LocalLLaMA/comments/1u795pb/donate_... We've…

There's `--filter=blob:none` and it allows to automatically fetch blobs when needed.

It's 2026. Historically the way for large binaries in git was git LFS. Now the way for large binaries in git is just git.