Any amount of latency seems like too much latency when jamming with others. I could see how this would work with audio engines like SuperCollider, but I'm curious how one goes about this for live recordings.
Your ISP and their ISP have a certain amount of delay or lag which can't be avoided, but it's possible to improve your delay on your own side of the cable / dsl / fiber modem by using Ethernet instead of Wifi. My Wifi router (a slightly older, last-model Apple unit) introduces about 10-12ms all by itself.