At its core, the main drawbacks that need to be solved for them to be a viable option are imo:
* Improving OS flows. Every passkey implementer that's also an OS gets really excited about enrolling you into their proprietary clouds, and using alternate flows to respect the users wish to use their own manager is usually hidden in confusing UI forms that don't feel consistent if you don't already know what you're doing.
* Device loss scenario is already mentioned, although more broadly speaking a lot of the reasons people get leery is because all three major providers (Google, Microsoft, Apple) are notorious for their near black box technical support. Losing access to one of these providers on its own is already enough to heavily disrupt the average person's life. Having your login details stored with them makes this even worse.
* The FIDO Alliance Is Way Too Excited About Device Attestation And I Don't Like It. Basically the FIDO Alliance's behavior around passkeys reeks of security theater and them badgering an open source password manager for daring to let users export their passkeys in the format they preferred, rather than what the FIDO Alliance wanted (which is that passkeys must always be encrypted with a password) is telling. If they are as secure as promised, it's a bad look to start threatening device attestation as a means to get people to comply with your specific idea of security. The only real barrier right now to it outright being a thing is that Apple zeroes out the field and when Apple is the only meaningful halt to that kind of attestation, something has gone very wrong.
I think passkeys are interesting, but I just flat out don't trust the FIDO Alliance with the idea. They're way too invested in big tech being good stewards of the ecosystem, which is becoming increasingly unpalatable as more and more evidence piles up that they're really bad actors on everything else. (So why should we trust them with our credentials?) The idea genuinely has value (it's literally the same kind of mechanism as SSH keys), but the hostility towards user freedom is deeply concerning and a blocker to getting people to use it. Even non-technical people seem leery of them, just because of how aggressively big tech has been pushing it.