I have been using matrix mostly as an IRC/Gitter client but also for a small number of non-technical friends. It is making excellent progress but has a long way to come. However I do think it is the right investment due to the direction and funding it has.
Some weak points (based on using element.io clients which has some features that aren't core matrix):
- Cross-session signing is not natural to set up for users, however this likely isn't much of an issue as most of these will just use it on their phone. However this can also be an issue if they want to switch device but lost their keys.
- e2e verification is a bit awkward. No great TOFU option which makes initial setup a bit awkward, you can do it but the UI doesn't encourage it. It would also be interesting to have the option to "share" your verification. So if friend X says they trust these keys for friend Y I can import Y without further verification.
- Occasionally new devices don't want to send messages to the remote user's session. So the user just gets a "Message encrypted" notification with no indication to me that I sent a message not encrypted to all of the user's sessions.
- Voice + Video is super basic for 1:1 calls. For >2 users it uses Jitsi which is fantastic but this different UX between the two is also confusing. I'm hoping that when Jitsi launches e2e encrypted calls it can just be used for everything.
- Visually the app is cluttered and not perfectly clear. For 1:1 calls "bubbles" such as used by most chat apps are super clear. Matrix uses a design better for many user chats but can be less clear. Especially replies look a bit odd in Element.
- URL previews are only supported on web and need to be turned on per-chat (if encrypted). This is for legitimate privacy reasons but is a sacrifice of UX.
Overall once you get it set up it just works and has all of the features that a user would expect. However I'm looking forward to a lot of the polish landing which will make it compelling on UX grounds alone, whereas now it just seems like the best compromise when you consider the privacy and decentralization features.