Hmm, perhaps not. We may be agreeing with each other. I have no desire to punish people for their crimes, only to remove them from society.
I thought your point was that the concept of prison as "a place where you are removed from society" is fundamentally flawed, and your rationale was that removal does not serve the person removed. It seems that your point is actually that the modern concept of prison as "a place where you go to be punished" is flawed, which I agree with. Prisoners should simply be outside society, not in pain or fear. If they can manage to live peacefully that ought to be allowed; if they cannot, they ought to be restricted further.
I think there are only two real "punishments", which are death and exile. Corporal punishment is just partially-applied death, and prison is just partially-applied exile. All other punishments I'm aware of are just shades of either death or exile; even monetary fines are "a reduction of your ability to participate in society".