One thing is that Powershell has some built-in "cmdlets" which would be buried in the filesystem (and not in the PATH) otherwise. I'm thinking specifically about the PS equivalent of signtool.exe, which is normally in an SDK path under program files (x86) depending on which SDK(s) you have installed.
Do I remember the name of the signing cmdlet(s)? Nope! Is this really better than just using alias, or changing the PATH? Probably not.
Another reason is vendor support. For example, VMware publishes a set of PS cmdlets as 'PowerCLI' for doing stuff in vSphere, but if you want to do something outside PS you're going to have to script/code against an SDK. I suspect there are other vendors doing the same thing. That may be a PS benefit. I usually end up writing a script instead although I found e.g. bulk movement of VMs from one set of datastores to another easier in PS.
The final thing I can think of is that there are a few situations where maybe you want things to stay in-process, and you get that w/ PS whereas you wouldn't with bash. An example I have is a script that facilitates remote deployment of an MSI with impersonation of a user on the remote machine. This is pretty straightforward with a WMIObject in PS, but not in cygwin bash since Windows doesn't really have sudo or su -c (runas only works interactively) and wmic doesn't offer any impersonation capability.
None of the above are really compelling for me, I end up spending 99+% of my time in bash/tmux/mintty.