I've just switched over to using the Yubikey Neo, so I'd love to chime in here.
My first and primary use of my key is that I use HMAC hashing on the Yubikey to unlock my KeepPassCX database. This solution works very well for me because it works seamlessly on multiple platforms (Linux and Windows) and is also compatible with Keepass2Android for my phone. I've looked into GPG only solutions and the ones I looked at didn't offer either the cross-platform compatibility and or browser integration, which is nice. The strong advantage of the Neo (vs the other hardware keys) is the use of NFC, which means less plugging things into my phone. In an ideal world, I'd love to get NFC working with my computers too.
I also use the PGP/SSH smartcard capabilities of my Yubikey on Linux (Ubuntu) and that's been flawless. I don't use SSH on Windows, but I hear the integration there is fairly solid as well.
I've not yet begun to use U2F for authentication, yet. My focus is on my most important services first, which is my passwords and logins. I'll be moving to U2F soon though.
If there was a way to store my passwords easily using PGP instead of HMAC, I'd be interested in that. The issue isn't the storage, but the interface to that storage being easy to use and cross platform (which is not the domain of the hardware, obviously). If I had that, I could consider not needing the HMAC.
I've heard that there is a way to use U2F offline for SSH, but I haven't looked into it. I'm still using SSH keys for things like SSH and Git. Perhaps if there was a solution to that, I might drop the GPG requirement, but I'd need either HMAC or GPG for decryption of my password database, so in any case, FIDO alone isn't enough.