Why do we add the positional encoding to the original vector instead of appending it? Doesn't adding positional data to the embedding destroy information?
The hope is that the neural network will learn to undo the positional information and understand that it's an augmentation used to tell the neural network where the word occurred in the sequence.
Sequence being a misleading term since the input to the network is actually a set of tokens meaning no positional information without the extra information added.