SSO is a great benefit to the customers, with real tangible security and management benefits.
I'm however speaking from the point of view of the service provider (the SaaS app) and about SAML in particular. I feel that the addition of SAML into a given service is a net-negative from that service's security point of view. It's a large additional complex attack surface, many open source SAML libraries that I've reviewed have a history (and in some cases open issues right now) of "pants on head" type of security errors. A popular library in use right now, has a known race condition where it gets confused if there are concurrent SAML requests happening.
And that's just the libraries. Then you have to use them correctly. The libraries do the absolute minimum checking since they don't have the context, you have to add a laundry list of your own checks to them. Just recently there was a HN article about taking SAML assertions posted to provider A and re-using them on provider B, where clearly the most basic of checks aren't in place at all. There's all kinds of confused-deputy type of problems I believe most service providers don't think about at all. And that was an easily offline checked attribute, I believe if you'd start to check how many services correctly implement even the basic "inResponseTo" check on SP-initiated flows (which requires a distributed cache on the service provider side), you'd find they don't.