Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Low-latency jamming over the internet

sub.live

Post reply on HN
Print view View original

111–120 of 124 posts

Re: Show HN: Low-latency jamming over the internet

#111

Post by rdtennent »

weepy wrote: post #98
rdtennent wrote: post #80

Earlier quoted context omitted.

"requires a central server" is misleading. You can set up a server wherever; there's no central server. Except for the multiple-ISP issue I discussed, there's no reason access to a server is more latency than access between the "clients". The advantage of the server/client model is that the clients can be very lightweight: I personally use a Raspberry Pi 3 with an ultra-low latency DAC/ADC hat. Works fine. All the re…

Well you do need to find a server midway between all the users, which is a hassle of course. I don't personally like the model either becuase it needs to decompress, mix and recomopress all the streams on the server and also needs an extra jitter buffer. The only clear benefit to it AFAIK is that it scales better for larger groups O(N) rather than O(N2)

"need to find a server midway between all the users" Nonsense. Did you read my post? The server is in Montreal; all the clients are 200 miles west of Montreal in Kingston. Physical distance isn't important. What is important is ping time. When one tries to connect in Jamulus, dozens of public servers are suggested with their ping times listed. If none are suitable or one wants a private server, one can set up a server anywhere that provides good ping times to all the intended clients. In the multiple-ISP scenario I had to deal with, a server "midway between all the users" would have been useless, as would any kind of server-less topology.

"The only clear benefit to it..."? As I suggested, a significant benefit of the Jamulus model is that any of the clients can be "thin"; most of the computation is done on the server. You may be able to improve on this but disparaging it with silly criticisms isn't going to help you; there are thousands maybe millions of satisfied Jamulus users around the world.

Re: Show HN: Low-latency jamming over the internet

#112

Post by retrac »

tomxor wrote: post #41

It's good that latency is considered to be so critical, but for the same reason I'm sceptical that this would work well for the majority beyond quite local ranges with very good internet connections i.e some kind of fiber... which most people don't have, although I realise a lot of the tech crowd is unaware of this (most of the worlds user end points are some kind of DSL or cell network with a 20-40ms minimum). If an…

True for many users. My mobile latency is ~100 ms, and that's still impressively quick to me. But I find DOCSIS to be very low latency where I live. My connection at home is ~10 ms to other home users on the same provider in this city, and generally within 20 ms anywhere in the southern part of the province and 40 ms most of the way across the continent. As long as they aren't on DSL or mobile on the other end, anyway.

Re: Show HN: Low-latency jamming over the internet

#113

Post by rdtennent »

weepy wrote: post #73
rdtennent wrote: post #57

My experience with Jamulus might be instructive. Members of our group live in the same city (Kingston, Ontario) but use two ISPs. Packets between users on the same ISP were fine but packets from one ISP to the other were being routed via Toronto and then Chicago and ultimately back to Kingston. It's not the distance travelled that's the problem (speed of light), it's the latency introduced by each intermediate node.…

The issue with Jamulus is that it requires a central server - which means it needs to be close to everywhere . It also needs double buffering and double compression. P2P is the way forward here IMHO.

> P2P is the way forward here IMHO.

But if any of the O(N^2) P2P links has unacceptable latency, eliminating the server would be counter-productive because the client-to-server connection may have less latency if the server is located appropriately.

Re: Show HN: Low-latency jamming over the internet

#114

Post by audray »

telesilla wrote: post #109
weepy wrote: post #93

Earlier quoted context omitted.

It uses WebRTC for the video, but the audio latency of WebRTC is too large and uncontrollable. As stated in the post, the audio uses a custom C++ UDP solution. As far as I know it's the first video calling app with very low latency audio.

How do you deal with firewalls in that case? Sonobus has a similar problem that if you don't have accessible NAT you can't connect. You need a relay or central server which can get really expensive!

Yeah one reason I like Jacktrip, you can optionally use servers to connect. Low latency is nice but it doesn't help in a performance when you can't connect to your group of musicians

Re: Show HN: Low-latency jamming over the internet

#115

Post by weepy »

rdtennent wrote: post #111
weepy wrote: post #98

Earlier quoted context omitted.

Well you do need to find a server midway between all the users, which is a hassle of course. I don't personally like the model either becuase it needs to decompress, mix and recomopress all the streams on the server and also needs an extra jitter buffer. The only clear benefit to it AFAIK is that it scales better for larger groups O(N) rather than O(N2)

"need to find a server midway between all the users" Nonsense. Did you read my post? The server is in Montreal; all the clients are 200 miles west of Montreal in Kingston. Physical distance isn't important. What is important is ping time. When one tries to connect in Jamulus, dozens of public servers are suggested with their ping times listed. If none are suitable or one wants a private server, one can set up a serve…

P2P doesn't require any computation on any server - it's essentially serverless. It doesn't make much sense to me at least to need to compress everything twice and have multiple buffers AND have to manage a server.

Re: Show HN: Low-latency jamming over the internet

#116

Post by rdtennent »

weepy wrote: post #115
rdtennent wrote: post #111

Earlier quoted context omitted.

"need to find a server midway between all the users" Nonsense. Did you read my post? The server is in Montreal; all the clients are 200 miles west of Montreal in Kingston. Physical distance isn't important. What is important is ping time. When one tries to connect in Jamulus, dozens of public servers are suggested with their ping times listed. If none are suitable or one wants a private server, one can set up a serve…

P2P doesn't require any computation on any server - it's essentially serverless. It doesn't make much sense to me at least to need to compress everything twice and have multiple buffers AND have to manage a server.

Well if any of the O(N^2) connections has unacceptable latency, you'd understand why a server is sensible. Reducing computation won't improve latency in the connections. One has no flexibility with clients but a server can be located wherever good connections are available.

Re: Show HN: Low-latency jamming over the internet

#117

Post by mandor »

iandanforth wrote: post #62

It would be fun to try to add a predictive layer to this: - Given the score and what each person has just played predict what the next few sounds are going to be - Given a high frame rate video stream of a person predict what the next note to be played In the same way that Nvidia has extremely low bandwidth but high resolution video enabled by face keypoint tracking and facial reconstruction / puppeteering maybe ther…

We're doing exactly this to teleoperate humanoid robots on high-latency networks!

Paper: https://arxiv.org/abs/2107.01281

Video: https://www.youtube.com/watch?v=N3u4ot3aIyQ

"We introduce a system in which a humanoid robot executes commands before it actually receives them, so that the visual feedback appears to be synchronized to the operator, whereas the robot executed the commands in the past. To do so, the robot continuously predicts future commands by querying a machine learning model that is trained on past trajectories and conditioned on the last received commands. In our experiments, an operator was able to successfully control a humanoid robot (32 degrees of freedom) with stochastic delays up to 2 seconds in several whole-body manipulation tasks, including reaching different targets, picking up, and placing a box at distinct locations."

Re: Show HN: Low-latency jamming over the internet

#118

Post by prendo1234 »

weepy wrote: post #106
prendo1234 wrote: post #84

Hey @weepy. Awesome project! As a webrtc dev - interested to hear about your choices around opus packet durations, FEC percentages, ARQ strategies, jitterbuffer length and packet redundancy.

I used the standard 2.5ms. I know you can go lower if you want, but then you need a higher bitrate as it's "custom". I turned off FEC as it adds latency. Jitter buffer right now is just user controlled. A bit lame, - I should make it automatic, but need to get the right heuristic. With a LAN connection, the buffer can be as low as one or two packets. I don't use any packet redundancy either and there's no ARQ as if y…

Yeah, awesome. Have to say this is the best version of something like this that I have tried. The packet capture didn't seem to be RTP, are you using libwebrtc under the hood?

The system seems to have the settings right for realtime collab. In fact I found it much better for chatting too if you can guarantee they have a headset and good mic - I hate current-gen AEC algorithms. Interested to see how it fares in a 3+ participant setting.

I had a go at jamming with my brother yesterday. We had to adjust down the delay to 2ms (I think default was 5ms) in order to counteract the "lagging" effect you get when you lock into the remote beat. Once we had that tuned it worked really well. We would occasionally suffer burst losses, but you can play through it and it's still synced afterward which is the best you can do with an unreliable network.

Some feature requests: - Join room by name (we were confused that there was a different code than the name we chose) - Auto register new audio devices when you plug them in/edit in settings. - An easier way to join with multiple audio inputs. We had trouble setting up our instrument, but also adding a mic for chatting. - Local recording of your end with some shared sync markers, that you could manually or automatically sync up post-hoc

The use case I use WebRTC for is slightly different, where quality >> latency , so we have it tuned to the other end and run the full gambit of FEC, ARQ, packet redundancy etc. Interesting to hear about other's solutions though!

Re: Show HN: Low-latency jamming over the internet

#119

Post by ad8e »

ad8e wrote: post #105
weepy wrote: post #74

Earlier quoted context omitted.

You must have set it up wrong. It doesn't add any latency, but you do need a higher bitrate to get similar quality.

I googled "opus packet loss" and found https://www.asterisk.org/asterisk-opus-packet-loss-fec/ , which says "an Opus encoder can embed redundant data about the preceding packet in-band in the current packet". That method would cause 1 packet of delay. Are you using something different, or avoiding the delay through another method like tiny packets?

Here's probably the answer: https://opus-codec.org/docs/opus_api-1.2/group__opus__decode...

"Lost packets can be replaced with loss concealment by calling the decoder with a null pointer and zero length for the missing packet."

Re: Show HN: Low-latency jamming over the internet

#120

Post by weepy »

prendo1234 wrote: post #118
weepy wrote: post #106

Earlier quoted context omitted.

I used the standard 2.5ms. I know you can go lower if you want, but then you need a higher bitrate as it's "custom". I turned off FEC as it adds latency. Jitter buffer right now is just user controlled. A bit lame, - I should make it automatic, but need to get the right heuristic. With a LAN connection, the buffer can be as low as one or two packets. I don't use any packet redundancy either and there's no ARQ as if y…

Yeah, awesome. Have to say this is the best version of something like this that I have tried. The packet capture didn't seem to be RTP, are you using libwebrtc under the hood? The system seems to have the settings right for realtime collab. In fact I found it much better for chatting too if you can guarantee they have a headset and good mic - I hate current-gen AEC algorithms. Interested to see how it fares in a 3+ p…

Hey thanks.

You can also send the backing track ahead of time a little so that you both hear it at the same time!

Do you mean that you want to mix the inputs from multiple devices ? This is possible - but you do get a little bit of extra latency of course.

It's supposed to refresh the audio devices when you "mouseenter" the select box, so perhaps there's a big there.

Multitrack recording with a bigger buffer would be great.

Good you found the buffer settings. I'm surprised that you found a significant difference though between 2ms and 5ms?

Post reply on HN