Hi, I have read your interesting paper. I recommend you our previous HyperZZW paper ( https://arxiv.org/pdf/2401.17948 ). I think there are a lot of similar concepts here. 1. Context-dependent convolution 2. Global & Local branches 3. Replace large-filter Conv with matrix multiplication 4. Information bottleneck -> Information loss I also want to share that Mamba is based on the concept of Hyena. And the simplicity i…
Thank you for your comment and for sharing your interesting work. I'll take a look.