Popular is not always (or even often???) the best way to go ;-) Being on the fringe usually means you've thought about it more than most people.
I'm an OO guy who has been moving more and more in the FP direction. I look at it a bit differently that you, but basically have the same conclusion (I think).
Rather than saying OO and FP are orthogonal, I tend to feel that OO and FP are essentially the same, but that popular OOP misses the point completely.
If I have a struct and a set of functions, where the first argument of the function is always the struct, how is that different than having an object with a set of methods? It might (or might not) be implemented under the hood differently, but whether I write the object on the left, or as the first parameter, it's all the same.
With respect to mutability, most FP languages don't allow mutable data structures. I can likewise restrict myself to immutable data structures in OO languages. Most people don't. Unless you have a specific performance issue in mind, I think this is usually a mistake. I think the fact that mutable data structures are popular, is not equivalent to saying the OO encourages mutable data structures. It's equivalently bad in both camps (which I suppose is the meaning of "orthogonal" ;-) ). Immutability has memory management considerations which I will discuss in a minute.
WRT polymorphism, with OO languages, ad-hoc polymorphism (operator overloading) is essential, while parametric polymorphism (generics) is optional. With FP, it tends to be exactly the opposite. But if you consider that both ad-hoc polymorphism and parametric polymorphism are useful tools in both approaches, I think it's kind of a moot point.
The rest of it boils down to coupling and cohesion. In OO, you have encapsulation which forces you to reduce your coupling in certain ways. You tend to have cohesive code in that all functions dealing with a particular data structure are grouped with that data structure. In FP, you have cohesive code in that all your functions doing a particular operation are grouped together. Coupling is not strictly enforced, but writers of good FP code enforce it themselves. But it's not like you can't do both styles in both systems. It's just that the language implementations make one style easier than the other.
The only place that I see which is very different is in garbage collection. If you are using immutable data structures, it is hard to know when to deallocate the memory. So you need some kind of garbage collection (even if it is just reference counting). Having it built in to the language makes it very convenient, but it is hardly a show stopper if you have to write your own (or use a library).
With C++ there is an idiom (which I still think is a good one) where you try to only allocate memory on the stack. Everything is a local variable and you send those local variables to functions that fill in the data for you. That way you can explicitly understand the lifetime of every data structure in the system. I've done a lot of embedded code where I needed to keep track of every byte of code and to know exactly where it is allocated and deallocated. Techniques like this help, but it is obviously not immutable. I don't know how to do this kind of thing with traditional FP languages.
But really, it's only the very last thing that strikes me as being particularly different. The rest is syntax and how much typing I need to do. It's more that the language designers have idioms that they prefer and make those idioms easier. FP and OO seem to share the same pool of idioms, though.