Viewing profile — charleshn

Thanks for the article. I have two questions/comments: 1. The N-ring fade-in is quite neat. I guess without the constraint of hash parity rendezvous hashing [0] could have been an …

> Some people like to parrot "next token prediction", "LLMs can only interpolate", and other nonsense Thank you for illustrating my point.

Yes, they can. Some people like to parrot "next token prediction", "LLMs can only interpolate", and other nonsense, but it is obviously not true for many reasons, in particular sin…

Not sure what you mean: OpenEvolve is an open source implementation of AlphaEvolve: https://huggingface.co/blog/codelion/openevolve

They'll likely make it available at some point, but for now one can use OpenEvolve [0] which is not quite as good but should be a good start to use the same LLM-driven evolutionary…

> Application pods learn the current assignment through a library called the Slicelet (S for server side). The Slicelet maintains a local cache of the latest assignment by fetching…

They should be reintroducing the 3D vcache [0] variants (X) in EPYC, with a higher cache/core ratio, that was present in EPYC4 (e.g. 9684X [1]) they for some reason wasn't availabl…

It's fundamentally because of verifier's law [0]. Current AI, and in particular RL-based, is already or will soon achieve super human performance on problems that can be - quickly …

> std::hardware_destructive_interference_size Exists so you don't have to guess, although in practice it'll basically always be 64. Unfortunately it's not quite true, do to e.g. sp…

> There's a good reason so much research is done on Nvidia clusters and not TPU clusters. You are aware that Gemini was trained on TPU, and that most research at Deepmind is done o…

I can relate. I had tinnitus for over 10 years. My tinnitus was not the usual ringing type, it was some sort of humming, low frequency noise. The frequency was not constant, it cou…

A few questions if the authors are around! > Is hardware agnostic and uses TCP/IP to communicate. So no RDMA? It's very hard to make effective use of modern NVMe drives bandwidth o…

Interesting that neither the article nor the comments mention the CALM theorem [0], which gives a framework to explain when coordination-free consistency is possible, and is arguab…

You can have a look at the DeepSeek paper, in particular section "2.2 DeepSeek-R1-Zero: Reinforcement Learning on the Base Mode". But generally the idea is that it's, you need some…

> We cannot add more compute to a given compute budget C without increasing data D to maintain the relationship. > We must either (1) discover new architectures with different scal…

Yes, 450GB/s is the per GPU bandwidth in the nvlink domain. 3.2Tbps is the per-host bandwidth in the scale out IB/Ethernet domain.

Could you check the value of your kernel's net.ipv4.tcp_slow_start_after_idle sysctl, and if it's non zero set it to 0?

You can now add getting gold at IMO [0] to the above list. [0] https://x.com/alexwei_/status/1946477742855532918

Frontier models went from not being able to count the number of 'r's in "strawberry" to getting gold at IMO in under 2 years [0], and people keep repeating the same clichés such as…

I mentioned algorithms, not software engineering, precisely for that reason. But the next step is obviously increased formalism via formal methods, deterministic simulators etc, ba…