There's value in being early - in the right thing. - If you'd invested in Bitcoin in 2016, you'd have made a 200x return - If you'd specialized in neural networks before the transformer paper, you'd be one of the most sought-after specialists right now - If you'd started making mobile games when the iPhone was released, you could have built the first Candy Crush Of course, you could just as well have - become an Acti…
> - If you'd specialized in neural networks before the transformer paper, you'd be one of the most sought-after specialists right now > - If you'd started making mobile games when the iPhone was released, you could have built the first Candy Crush I disagree: Concerning the first point: how neural networks are today is very different from how they were in former days. So, the knowledge of neural networks from the pas…
If you had worked on anything other than a transformer based architecture post 2016, such as Mamba or RWKV, you would have wasted your time.
Mamba 3 is the third iteration and somehow I doubt that it will catch on.