> Why should neighbors get a say in your legal activities?
This is one reason why there are different layers of rules on what a property owner can do - zoning laws, building codes, ordinances, community rules. Neighbors get a say because what you do on/with your property can significantly affect your neighbors. When one property owner does something that adversely affects a neighbor, that sometimes leads to conflicts which play out in courts or councils.
If you want examples of how neighbor disputes result in new rules, just do some searching for "neighbor dispute". You'll find some amusing, scary, and even baffling cases.
And with regards to living in close proximity to others, that is something that you know before you choose to live somewhere. It's not usually something that suddenly changes. You may have a quiet neighborhood with relative privacy. When your neighbor uses their back yard, it's perfectly reasonable. You know or quickly learn how many people live on each side of you, and you acclimate to the level of traffic/activity around you. And judging by your research and what you experienced during early viewings of the property, you decided whether you were comfortable with that level of activity. But now your neighbor(s) are adding ADUs, potentially doubling the number of people living around you (and correspondingly doubling the activity and traffic).
> I would certainly hope so, these are human beings not indentured servants
Do your neighbors normally walk between the houses to enter their own house via the back door? Probably not. Will the ADU renters be walking through their landlord's house to reach their ADU in the back? Probably not. And chances are, your house is fairly close to your neighbor's house.
Prior to the ADU situation, you probably didn't have much foot traffic, particularly late at night, alongside your house (between yours and your neighbor's houses. But now you have one or more people walking between the houses, by your bedroom window, as often and at whatever time as they want. You might have made a difference decision regarding buying your house if you had known this would be the case.
This is a bit similar to the hoopla when Waze first became popular. Occasionally a quiet neighborhood with small, low traffic roads would become a major thoroughfare because Waze routed people around large road traffic and through your neighborhood. Residents were quite disturbed.
Now think about property crime. It's very easy to steal or vandalize property, but it isn't a problem in many areas because the people living in those areas become familiar with their neighbors and people's patterns of behavior. And when someone notices something peculiar, they may contact a neighbor (or the police) to ensure that everything is ok.
Increase the population in the neighborhood (and especially add new traffic paths which specifically involve entering people's back yards), and you have greatly increased the difficulty of residents having a sense of what activity is normal and what is potentially criminal.
Yes you can return to the apartment or high occupancy argument, but areas of high occupancy have already adapted or built in behaviors to help prevent crime. Usually it involves more walls, less windows, more locks, more isolation. These aren't reasons why people typically buy a home with a yard.
I frankly don't care if people do this ADU thing or not. I'm just pointing out that it will be a change with significant impact that many people will not expect. This will change the character of neighborhoods from places where people settle and live to places landlords buy because of the rental opportunities. You'll have higher turnover of people in the neighborhood. It will essentially turn a traditional neighborhood into an apartment block with extra lawns. I promise you'll see groups with good financing buying up these homes to rent (and to rent the ADUs in the back) more than you'll see regular (non-landlord) people coming to enjoy the kind of life that apartments don't provide.