Viewing profile — cgdl

Very cool. For the INT4 QAT model, what is the recommended precision for the activations and for the key and values stored in KV cache?

Which model does the demo use?

I'd say llm inference requires both memory capacity and bandwidth. Cerebras provides bandwidth with on-chip SRAM, but not capacity (an entire wafer has only 44GB SRAM).

Indeed, and even if the cost per wafer was 300K, since about say 20-50 wafers are needed, its still 6MM to 15MM for the system. So likely it would appear this is VC subsidized.

Do you distinguish betwen "chips" and the wafer-scale system? Is the wafer-scale system significantly less than 3MM? EDIT: online it seems TSMC prices are about 25K-30K per wafer. …

Exactly what I was thinking. What sort of latency do you think one would get with 8x B200 Blackwell chips? Do you think 1500 tokens/sec would be achievable in that setup?

Do we know how far this event was from earth? Wouldn't that distance be the determiner of what the relative contraction observed on earth would be?

I recently came across a critique of the Turing test that seems relevant here. Given the test's limited duration (five minutes in this study) and the constrained rate of human comm…

Thank you for the great discussion. You've put your finger on the right thing I think. We can now dispense with the old VC-type thinking (i.e., that it's because the hypothesis spa…

Thank you, this makes sense. I am thinking of this as an abstraction/refinement process where an abstract notion of the longer completion is refined into a cogent whole that satisf…

Thank you. In my mind, "planning" doesn’t necessarily imply higher-order reasoning but rather some form of search, ideally with backtracking. Of course, architecturally, we know th…

Yes, and that's the problem. What Zhang et al [2] showed convincingly in the Rethinking paper is that just focusing on the hypothesis space cannot be enough since the same hypothes…

Agreed, but PAC-Bayes or other descendants of VC theory is probably not the best explanation. The notion of algorithmic stability provides a (much) more compelling explanation. See…

Very interesting. A related paper from a couple of years ago proposed a similar idea to understand generalization in deep learning: https://arxiv.org/abs/2203.10036