Earlier quoted context omitted.
Taking pictures wouldn't be illegal from a public place, similar to google's street view. Taking multiple pictures over a prolonged time could likely fall under harassment laws, depending on your state or country. Videoing someone else's property, if done on purpose, is illegal, unless you have a valid reason for doing so. If your neighbour has 'violated the peace', IE noise disturbance (that you called the cops for)…
In the US at least, photography on public property does not require the permission of the subject, otherwise photography in crowded tourist spots would be well nigh impossible. There are of course exceptions, such as when the subject expects privacy (even in public, upskirts are no good), when you are making a profit from your photographs (which will then require model releases), or when "national security" is involv…
I actually mistyped that sentence, sorry. What I meant was: I would only photograph something I own, or that no one owns.
I don't know how I messed that sentence up, I guess my brain flat-lined for a minute.