Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Native Matrix VoIP with Element Call

element.io

Post reply on HN
Print view View original

121–130 of 130 posts

Re: Native Matrix VoIP with Element Call

#121

Post by Arathorn »

anticensor wrote: post #120
Arathorn wrote: post #116

Earlier quoted context omitted.

Mixed sync +/- 100ms of latency or whatever is probably fine, for most purposes? If you really want everyone to be centralised on a single SFU then you’d set the permissions on the room appropriately.

It can be distributed SFU just the relative timings need to converge deterministically (possibly by servers agreeing on a virtual apex to artificially delay the streams to make up for topological differences) to make everyone hear the same thing.

So if it turns out that the streams need to be synced then we can delay them to do so - but i am a bit unsure on how important that really is. Are you worried about folks talking over each other if they think they successfully interrupted but then a remote user thought the same and they collide and backoff, like csma/cd?

Re: Native Matrix VoIP with Element Call

#122

Post by anticensor »

Arathorn wrote: post #121
anticensor wrote: post #120

Earlier quoted context omitted.

It can be distributed SFU just the relative timings need to converge deterministically (possibly by servers agreeing on a virtual apex to artificially delay the streams to make up for topological differences) to make everyone hear the same thing.

So if it turns out that the streams need to be synced then we can delay them to do so - but i am a bit unsure on how important that really is. Are you worried about folks talking over each other if they think they successfully interrupted but then a remote user thought the same and they collide and backoff, like csma/cd?

> Are you worried about folks talking over each other if they think they successfully interrupted but then a remote user thought the same and they collide and backoff, like csma/cd?

Yes, that is really important for classroom use cases.

Re: Native Matrix VoIP with Element Call

#123

Post by 7sidedmarble »

anticensor wrote: post #82
Arathorn wrote: post #50

Earlier quoted context omitted.

The competitive gap with Discord in terms of media quality is probably something like: * Need a low-latency SFU. This should be very doable; not only are there a lot of good FOSS SFUs to build on top of these days, the history of the Matrix team is actually that we built VoIP stacks fulltime before we shifted focus to Matrix, and we've built MCUs and media servers of all flavours in the past. MSC3401 should also give…

Discord voice is actually a server muxing MCU emulating an SFU.

Do you have any details on that? From what I've seen they just say it's a normal SFU architecture.

Re: Native Matrix VoIP with Element Call

#124

Post by throwaway29879 »

Arathorn wrote: post #113
throwaway29879 wrote: post #107

Earlier quoted context omitted.

Can you at least let us know what "stacks" anyone involved has produced? last I checked nobody involved has ever been involved in real voice platforms (as evidenced by 8 years of "voice first" but actually producing very little actual voice integration)

Sure. The entirety of the original Matrix team used to be a team called “next gen telephony” inside a startup called MX Telecom, which then got acquired by Amdocs and turned into their Unified Communications division, which I ran. The product was a horizontally scalable SIP softswitch built on resiprocate (B2BUA SBC and stateless router components), and then with media processing done using a C++ media graph framewor…

That's pretty much what I thought, which makes me wonder why at least SIP integration wasn't early on the list, there is the voip bridge but it relied on freeswitch and obviously is unmaintained so it doesn't work anymore, is there anything on the road map to make this happen?

Re: Native Matrix VoIP with Element Call

#126

Post by ryukafalz »

Arathorn wrote: post #50
jfkimmes wrote: post #44

Earlier quoted context omitted.

Could you give some insights on how you estimate the amount of effort of making Element Call competitive, performance-wise, with say Discord. I heard that Discord threw a lot of time and money at optimizing voice in their product. Can you just jump in and realistically compete? There are a lot of performance/latency/sound quality comparisons online of Mumble vs TeamSpeak vs Discord and recently Jitsi vs MS Teams vs Z…

The competitive gap with Discord in terms of media quality is probably something like: * Need a low-latency SFU. This should be very doable; not only are there a lot of good FOSS SFUs to build on top of these days, the history of the Matrix team is actually that we built VoIP stacks fulltime before we shifted focus to Matrix, and we've built MCUs and media servers of all flavours in the past. MSC3401 should also give…

> We're also in the process of adding in spatial audio (unsure if Discord has that) which should help a tonne with distinguishing the different audio feeds.

Thank you! This is something I've wanted in a videoconferencing app since I discovered it was a thing, and I'm glad it's Element doing it.

Re: Native Matrix VoIP with Element Call

#128

Post by pthatcherg »

Arathorn wrote: post #50
jfkimmes wrote: post #44

Earlier quoted context omitted.

Could you give some insights on how you estimate the amount of effort of making Element Call competitive, performance-wise, with say Discord. I heard that Discord threw a lot of time and money at optimizing voice in their product. Can you just jump in and realistically compete? There are a lot of performance/latency/sound quality comparisons online of Mumble vs TeamSpeak vs Discord and recently Jitsi vs MS Teams vs Z…

The competitive gap with Discord in terms of media quality is probably something like: * Need a low-latency SFU. This should be very doable; not only are there a lot of good FOSS SFUs to build on top of these days, the history of the Matrix team is actually that we built VoIP stacks fulltime before we shifted focus to Matrix, and we've built MCUs and media servers of all flavours in the past. MSC3401 should also give…

Hey, I'm the author of the Signal blog post about SFUs. I have a few questions/comments:

1. I don't think there are many good open source SFUs to choose from. I know of 2, maybe 3 (including our new one). There may be many, but few have good rate control. But maybe I just don't know about them? I'd be happy to learn of more good ones.

2. Echo cancellation is certainly a hard problem, but it doesn't conflict with E2EE unless you do it on the server, which isn't necessary. So perhaps it may be a somewhat harder problem because you close off one possible approach (doing it server-side), but many (most?) echo cancellation solutions are done client-side.

3. You may not be completely dependent on WebRTC's echo cancellation any more because of the new MediaStreamTrackProcessor and MediaStreamTrackGenerator APIs. I don't know if it will work for echo cancellation, but it might.

Re: Native Matrix VoIP with Element Call

#129

Post by Arathorn »

pthatcherg wrote: post #128
Arathorn wrote: post #50

Earlier quoted context omitted.

The competitive gap with Discord in terms of media quality is probably something like: * Need a low-latency SFU. This should be very doable; not only are there a lot of good FOSS SFUs to build on top of these days, the history of the Matrix team is actually that we built VoIP stacks fulltime before we shifted focus to Matrix, and we've built MCUs and media servers of all flavours in the past. MSC3401 should also give…

Hey, I'm the author of the Signal blog post about SFUs. I have a few questions/comments: 1. I don't think there are many good open source SFUs to choose from. I know of 2, maybe 3 (including our new one). There may be many, but few have good rate control. But maybe I just don't know about them? I'd be happy to learn of more good ones. 2. Echo cancellation is certainly a hard problem, but it doesn't conflict with E2EE…

1. So the SFUs we're currently looking at are yours, ion-sfu (and/or galene) and mediasoup. Honestly we haven't finished looking at how they compare for rate control, but the Pion team seems very interested in ensuring they have good rate control.

2. From context I think you're talking about noise cancellation here? I assumed that some of the more exotic ML-based ones ran serverside, which obviously is incompatible with E2EE. It sounds like there are a bunch of options for running WASM-based intelligent noise cancellation clientside though, especially with MediaStreamTrackProcessor and friends. rnnoiseless as a pure Rust->WASM port of rnnoise looks fun, for instance: https://github.com/jneem/nnnoiseless

3. True, although given Google are highly motivated to make AEC work properly in WebRTC, I guess I'm hoping that they'll continue improving it, much as they have been. I certainly never want to have to write or integrate one ever again :D

Re: Native Matrix VoIP with Element Call

#130

Post by arianvanp »

Arathorn wrote: post #48
arianvanp wrote: post #31

Earlier quoted context omitted.

Mesh doesn't scale for large conference calls though. Usually some SFU needs to be in play. But that's only possible if you have Insertable Streams support which for now only Chrome has

Safari DP 141 actually released RTCRtpScriptTransform support 2 days ago (thanks to https://trac.webkit.org/changeset/270107/webkit/ ). But we're waiting eagerly for Firefox to add it in https://bugzilla.mozilla.org/show_bug.cgi?id=1631263 . Hopefully by the time we've sorted out the SFU component, all the browsers will have RTCRtpScriptTransform implemented and so we'll get good cross-platform E2EE for larger calls…

Oh very nice!
Post reply on HN