Viewing profile — jul8234

Over the last years, state-tracking tasks, particularly permutation composition, have become a testbed to understand the limits of sequence models architectures like Transformers a…

https://openreview.net/forum?id=nvb60szj5C Twitter / X: https://x.com/julien_siems/status/1905628609714286687 Authors: Julien Siems , Timur Carstensen , Arber Zela, Frank Hutter, M…

Abstract: Linear Recurrent Neural Networks (LRNNs) such as Mamba, RWKV, GLA, mLSTM, and DeltaNet have emerged as efficient alternatives to Transformers in large language modeling, …

Abstract: Generalized Additive Models (GAMs) have recently experienced a resurgence in popularity due to their interpretability, which arises from expressing the target value as a …

Why not use something like LDA and features taken from word embeddings? A more probabilistic analysis would give more meaningful results.