This is a poor analogy because slavery wasn't a measure of well-being.
Slavery was the omission or suppression of freedoms and privileges that were considered rights. Slaves were bought and sold, and didn't have the right to choose to work (or not), or for whom. Slaves were not afforded the right to participate in democratic process.
Middle-class Americans are not restricted the way slaves were. A middle-class American is not restricted by law from deciding not to work, or for pursuing other career opportunities. They are allowed to vote. They are allowed to travel freely, and choose where they will live. To own property.
The thing that truly separated slaves from free people was not an income gap. It was that slaves were legally restricted from living like free people.
The concern will rise if and when middle class people are truly institutionally restricted from participating in society. If they were to institute policies to say only people with net worth over some value, or only landowners were allowed to vote. Or if laws were passed solely based on the lobby of big business to the detriment of the majority opinion. Or if people were legally restricted from changing jobs, or forced by law to work.
Some of these things are actually coming to pass. Particularly the manipulation of government and law by the very wealthy. The greater the income gap becomes, the more legislative power the wealthy have. The more legislative power the wealthy have, the easier it becomes for them to change the laws to maintain that power. When that power becomes enshrined in law, then we can start comparing the middle class to slaves.
We're not there yet, but I think there are people with a lot of wealth who would much rather see us there than lose their wealth.