So, I'm co-editor of the ActivityPub spec, hi! And yes, there's both reasons to want a Delete operation and to be skeptical of it!
Reasons to be skeptical of it: well there's the obvious ones, you might want to hold on to some content coming your way and lots of people are disappointed to see things disappear. Mutating networks have some dangers for sure.
Reasons it to want it, or that it's there: well, this is how all the existing federated social networks work. ActivityPub was designed in the context of standardizing basically something that projects like Mastodon, Diaspora, MediaGoblin, GNU Social, etc can use. And that is how things work now... mutation and federated deletion has been part of the existing federated social web for a decade. And we're trying to build something people can hook into their existing systems. People may also change their minds about things. Should they have the authority to remove their own posts? An open question, but at the very least, whether a server removes the content or not, a Delete is a request to do so across the network.
However, yes, there is an alternative structure one can imagine for these things, and that's something a bit closer to something git-like in the way that branches point at commits, but may change at which commit they point to over time, but the commits themselves never change. That's a desirable thing to explore, and I've blogged about it: http://dustycloud.org/blog/an-even-more-distributed-activity...
And we're even looking at exploring it, possibly, in the Social Web Incubator Community Group: https://www.w3.org/wiki/ActivityPub_extensions#Possible_futu...
BTW, if you're interested in such things, the community group is open to everyone and is hosting weekly meetings https://www.w3.org/community/swicg/ and we'll be hosting a meeting next week https://www.w3.org/wiki/SocialCG/2017-05-31