>But why is your experience tied to one brain and not another.

Because of the evolution. Life evolved from single-celled organism to multicellular organism building on top of individuals. Individuals which lived and cooperated in one community and one habitat evolved individually while living and reproducing together as one specie spreading genes to their offspring. There is no shared brain or shared consciousness in living beings, brain and consciousness are individual properties.

Something that is somewhat close to shared brain or shared consciousness is collective intelligence.