Viewing profile — pfekin_2nd

This is brilliant, thanks.

Author here — a few clarifications up front: How is this different from Performer / linear attention? Performer and related methods approximate the softmax kernel with random featu…

Summation-based aggregation replaces pairwise similarity with position-modulated projections and direct summation, reducing per-layer cost from quadratic to near-linear. On its own…