It could not be done peacefully. Any secession in the US will result in civil war. The two possible exceptions to this are Hawaii and Texas, since both have somewhat legitimate, but still legally controversial, justifications for secession.
But depending on circumstances, it could be a quick, unbalanced war. The US has the largest, most powerful military in the world, and that's only possible because of the union of all 50 states. The military has a huge interest in keeping the union together, and no group of states can hope to stand up to its might.
Active-duty service members from seceding states will not be universally for secession, but you can guarantee nearly every service member from other states will be against it. Even among those who support it, a good number will follow orders to suppress it.
In such a scenario, the only defense seceding states have is the threat of nuclear warfare. The case for Scottish independence would be similarly dangerous if Scotland were not willing to abandon their nuclear weapons. US states seceding cannot do the same because they would be immediately conquered by the nuclear states.
Speaking of, if secession did succeed, we'd be left with 2 nuclear states antagonistic to each other sharing borders. It would be India-Pakistan, but with many, many more nukes. A bad situation all around.
But again, I don't think there's a popular case for secession. It's not like e.g. the people of California hate the people of Tennessee and no longer want to be in a political union with them. There is a lot of hated between political groups, but the borders of that hatred are not state borders. Liberal voters in California hate those running and supporting the government in Tennessee (and vice-versa), but they don't hate the liberal voters in Nashville, Tennessee's capital. The split is mostly urban-rural, not state vs. state.
It creates a situation where we may hate those living only a few km away, but sympathize with the plight of our fellow American thousands of km away. And again, a lot of those people are our friends and family. We don't want to just abandon them.
> surely nobody would feel it worthwhile to fight a war over people in other states having the right to vote?
The US civil war was fought in part by those who felt those being enslaved in other states was an injustice. Modern war is much more catastrophic, but starting civil wars for the rights of our fellow citizens in other states is an American tradition. For all the reasons I hope there won't be war, this is the one thing that makes me fearful another war might start.