Intuition and Implementations for Gated Linear Networks
aiwabdn.github.io
Intuition and Implementations for Gated Linear Networks
1–6 of 6 posts
Re: Intuition and Implementations for Gated Linear Networks
#2This blog post details our interpretation of Gated Linear Networks, a new family of neural networks proposed by researchers at DeepMind. Implementations in JAX, NumPy, PyTorch and Tensorflow are here: https://github.com/aiwabdn/pygln
Re: Intuition and Implementations for Gated Linear Networks
#3This blog post details our interpretation of Gated Linear Networks, a new family of neural networks proposed by researchers at DeepMind. Implementations in JAX, NumPy, PyTorch and Tensorflow are here: https://github.com/aiwabdn/pygln
What prevents you from scaling it up and achieving good results on Imagenet?
Re: Intuition and Implementations for Gated Linear Networks
#4This blog post details our interpretation of Gated Linear Networks, a new family of neural networks proposed by researchers at DeepMind. Implementations in JAX, NumPy, PyTorch and Tensorflow are here: https://github.com/aiwabdn/pygln
What prevents you from scaling it up and achieving good results on Imagenet?
Main reason for now is that the cosine similarity used as context function is not suitable for natural images like those in Imagenet. We are experimenting some extensions based on convolutions for such use cases.
Re: Intuition and Implementations for Gated Linear Networks
#5Earlier quoted context omitted.
What prevents you from scaling it up and achieving good results on Imagenet?
Main reason for now is that the cosine similarity used as context function is not suitable for natural images like those in Imagenet. We are experimenting some extensions based on convolutions for such use cases.
What would you say is the best real world application for this class of networks?
Re: Intuition and Implementations for Gated Linear Networks
#6Earlier quoted context omitted.
Main reason for now is that the cosine similarity used as context function is not suitable for natural images like those in Imagenet. We are experimenting some extensions based on convolutions for such use cases.
What would you say is the best real world application for this class of networks?
Possibly some scenarios in RL where online learning is involved