"Firstly and most importantly, the stock market is not zero-sum."
It is, unless you have another definition of zero-sum. The talk about companies "growing in value" ignores the fact that the "value" is purely what the market will pay for those companies. If the market is a closed system, i.e. companies are not being listed or delisted, then the net profit made by buyers & sellers if all transactions were to be closed out is zero, by simple summation. (Actually, it's negative, because of significant transaction fees.)
"If your friend sells you a stake in his company, you have an ad-hoc stock market (a buyer and a seller for company equity). Is one of you destined to lose in this deal? Or can the company do well and you both succeed?"
If the company does well and you sell the stake to a third person (C), then you've profited and C is left holding the bag. Every single trade that is closed out results in either a profit or loss to its participant, and it's only the fact that the more money is put into the stock market over time that obscures the fact that, if all extant trades were to be closed out at one point in time (i.e. everyone cashed out), the net profit of everyone involved would be zero.
The market keeps growing because new money and participants are flowing in, but if this stops (say, due to foreign investors wanting to put their money elsewhere, or a shrinking investor population, or a recession), then all you have is a moribund market into which existing investors dare not put more money. The average profit for trades goes to zero because the average stock price is not moving up or down.
This is practically illustrated whenever a bubble bursts and money leaves the market: many participants start taking the losses that had been hidden by the previously rallying prices. The winners are the ones who got out first and took profits from those inflated prices.
I argued this point with two friends of mine, one who was an investment advisor, and another who was an oil trader. The advisor said that it wasn't zero sum and the market was always expanding, but after a bit of discussion and graph-sketching, he came to see that it was zero-sum after all. The oil trader (a pretty savvy chap) said, "Yeah, that's obvious. That's the game I play every day."
(I think this can be extended further to encompass the various financial markets as alternative sectors of a giant investment market offering various classes of product. In the end, each market is zero-sum, and so the overall investment market is zero-sum, which is obscured by the increase in production as technology advances. But this part is more speculative so I won't push it.)
I'm a bit occupied atm so I'll reply to the rest of your message if there's still interest later.