Viewing profile — fblgit

one of a kind single-transformer block layer, high throughput. The new generation of transformer-based lightweight models for common NLP tasks?

doesn't require much data, in a 7B can take a couple hours ~

Correct. UNA can align the MoE at multiple layers, experts, nearly any part of the neural network I would say. Xaberius 34B v1 "BETA".. is the king, and its just that.. the beta. I…

UNA: Uniform Neural Alignment. Haven't u noticed yet? Each model that I uniform, behaves like a pre-trained.. and you likely can fine-tune it again without damaging it. If you chat…