Another issue is what privacy are you losing by handing FB or Google all of your authentication? Offloading this is a huge privacy fail. It probably is a security win, but it's a huge privacy fail. Here Google/FB/etc, get MORE information for your giant catch-all, know-all database, thanks! Unfortunately better alternatives that are not a security win don't really exist yet.
What privacy are you losing? Real question. You're probably revealing to FB or Google that you're a user of web app X, but beyond that? Does using FB or Google auth enable any additional tracking of activities within the app or site? I would think only if the site developer was using Facebook or Google ads or tracking anyway. Asked another way -- if I sign into a website using FB auth, am I also signing into FB itsel…
But just the fact that an app uses Google authentication doesn't give Google any kind of privileged access to that app's data (beyond purely the data that they're logging in on this browser at this point in time) unless the app is pushing data actively back to Google (which they could do anyway for a gmail-managed email address, I guess) or you believe that Google will then subsequently forge authentication requests to that app and pull data out itself. Both of these things are hypothetical violations of your privacy enabled by using auth-with-Google, but for most use cases are rather unlikely, I would guess.