Live data from Hacker News

Native Matrix VoIP with Element Call

element.io

111–120 of 130 posts

Re: Native Matrix VoIP with Element Call

#111

Earlier quoted context omitted.

Something wrong with simply using RNNoise?

mmm, https://github.com/jitsi/rnnoise-wasm looks like it might help. wonder if jitsi actually uses it.

We are currently using it for the “your microphone is noisy” notification feature, not for actual noise detection.

With Audio Worklets or insertable streams, however, it should be possible to apply it to the stream im real time, though we haven’t tried it yet.

Re: Native Matrix VoIP with Element Call

#113

Earlier quoted context omitted.

The competitive gap with Discord in terms of media quality is probably something like: * Need a low-latency SFU. This should be very doable; not only are there a lot of good FOSS SFUs to build on top of these days, the history of the Matrix team is actually that we built VoIP stacks fulltime before we shifted focus to Matrix, and we've built MCUs and media servers of all flavours in the past. MSC3401 should also give…

Can you at least let us know what "stacks" anyone involved has produced? last I checked nobody involved has ever been involved in real voice platforms (as evidenced by 8 years of "voice first" but actually producing very little actual voice integration)

Sure. The entirety of the original Matrix team used to be a team called “next gen telephony” inside a startup called MX Telecom, which then got acquired by Amdocs and turned into their Unified Communications division, which I ran. The product was a horizontally scalable SIP softswitch built on resiprocate (B2BUA SBC and stateless router components), and then with media processing done using a C++ media graph framework we created called mxmedia. This was similar to gstreamer, but with more flexible threading and using C++ typing extensively to model media formats and propagate them through the media graph. It ended up with roughly the same featureset as today’s gstreamer, but about 10 years earlier - so with muxers, demuxers, packetizers, codecs, resamplers, colourspace transforms, AEC, AGC, rate control, SRTP, ICE, etc. We used it serverside for media gateways for transcoding calls for ISDN, H.323 and IAX; we then ported it clientside when smartphones came around to use it as a webrtc-style softphone SDK.

Unfortunately the IP was all proprietary and owned by the company, and when WebRTC stole our lunch we shifted focus to messaging and building Matrix, and have slowly returned focus to VoIP - hence Element Call.

Re: Native Matrix VoIP with Element Call

#114

This is really cool. I really feel like this ecosystem is hitting its stride. The list of upcoming features is long and sometimes I don't understand how they can have so many things going on at once. Features tend to take a few years to mature, but that is not a bad thing. I will be setting up my own server shortly, something that I have been thinking about for a while and maybe try to develop something. Congrats to…

https://gitlab.com/ptman/matrix-login this could ofc be modified to support oidc

Re: Native Matrix VoIP with Element Call

#115

Slightly off topic: I really like the idea of Matrix/Element, but I’ve found I can’t rely on it. In three instances, I’ve had a long-running chats stop working completely and simply not load. All history/media inaccessible. Just a loading spinner. This is Element on iOS. I wish it were more dependable, because things like this would excite me much more.

We have had some fairly nightmarish bugs on iOS thanks to how iOS changed push notifications a while back, meaning that you end up with a tiny client in the push extension trying to independently sync and decrypt e2ee traffic which was at risk of clashing with the main app and causing problems. I think I know the bug you are describing, and it was fixed several months ago - as of the current build the spinner should only show for less than a second and never get stuck on. Sorry that you got bitten by this; I’m also an iOS poweruser and the fallout from the push changes was painful.

Seperately, we wre also experimenting with a full rewrite of the iOS app on top of matrix-rust-sdk called Element X, which whould provide a much better and faster codebase to try to avoid this sort of thing in future.

Re: Native Matrix VoIP with Element Call

#116

Earlier quoted context omitted.

> I wonder if this setup can be leveraged to create a Twitch-like streaming solution based on Matrix. Yes, we've built it with this in mind. There are three possible approaches: * You can broadcast a headless client via HLS or RTMP, similar to how Jibri works for Jitsi, and how we broadcast FOSDEM ( https://matrix.org/blog/2021/02/15/how-we-hosted-fosdem-2021... ). Basically you run a headless Chrome against a virtua…

> If there are two, then folks will pick the one with the lowest latency. No it needs to be deterministic. "Just picking the lowest latency node" would result in sync failures i.e. not everybody hearing the same thing.

Mixed sync +/- 100ms of latency or whatever is probably fine, for most purposes? If you really want everyone to be centralised on a single SFU then you’d set the permissions on the room appropriately.

Re: Native Matrix VoIP with Element Call

#117

Earlier quoted context omitted.

So in the initial beta we haven't turned on E2EE, purely because it will make it way harder to debug any problems which surface. However, at this rate, things are looking pretty stable and i'd expect us to enable it in the next few weeks. It uses normal Matrix E2EE, which means a Double Ratchet between the pairs of devices participating in a given conversation, which is then used to secure the signalling which is use…

Will E2EE be on by default?

yup

Re: Native Matrix VoIP with Element Call

#118

Slightly off topic: I really like the idea of Matrix/Element, but I’ve found I can’t rely on it. In three instances, I’ve had a long-running chats stop working completely and simply not load. All history/media inaccessible. Just a loading spinner. This is Element on iOS. I wish it were more dependable, because things like this would excite me much more.

We have had some fairly nightmarish bugs on iOS thanks to how iOS changed push notifications a while back, meaning that you end up with a tiny client in the push extension trying to independently sync and decrypt e2ee traffic which was at risk of clashing with the main app and causing problems. I think I know the bug you are describing, and it was fixed several months ago - as of the current build the spinner should…

Wow! Seems it was user error, since I just updated it and it’s fixed. Thanks so much.

I wish I could update my parent comment. Although this interaction speaks for itself I think. Thanks for a great product!

Re: Native Matrix VoIP with Element Call

#119

I feel if XMPP just had some better tools this push for Matrix would be unnecessary.

If IRC had some better tools then the push for XMPP would be unnecessary. In fact, why didn't we just build IM on top of email? Turns out its much much easier to start from scratch with the right ideas than to transform an existing protocol in to something usable and convince everyone else and update all software to match. "The market" has decided that Matrix works well and XMPP doesn't. If it made sense to fix XMPP…

Someone did build IM on top of email: https://delta.chat

Re: Native Matrix VoIP with Element Call

#120

Earlier quoted context omitted.

> If there are two, then folks will pick the one with the lowest latency. No it needs to be deterministic. "Just picking the lowest latency node" would result in sync failures i.e. not everybody hearing the same thing.

Mixed sync +/- 100ms of latency or whatever is probably fine, for most purposes? If you really want everyone to be centralised on a single SFU then you’d set the permissions on the room appropriately.

It can be distributed SFU just the relative timings need to converge deterministically (possibly by servers agreeing on a virtual apex to artificially delay the streams to make up for topological differences) to make everyone hear the same thing.
Post reply on HN