Viewing profile — anarmorarm

Edit: GPT-2, not GPT-2 Medium. The 2nd paragraph should read: "With 23.8 PPL on WikiText-103, WaveletLM beats both GPT-2, which was trained on 80× more data, and Transformer-XL Sta…

WaveletLM is a wavelet-based, attention-free architecture that replaces self-attention with learned lifting wavelet decomposition, a Fast Walsh-Hadamard Transform, per-scale gated …

This deserves to be on the front page. Fantastic read.