Very very good, a little bit of stuttering during the honking I think but I like it overall! :D :) Jitsi Meet has been a great alternative to other meeting apps in these crazy times.
My experience with Jitsi Meet has been quite bad. My previous employer was a cheapskate, and they self-hosted Jitsi Meet. Random disconnections and instability were pretty much a daily occurrence, some people were disconnected every few seconds. While I suppose the self-hosting by Cheapskate Inc. was the main culprit, Jitsi’s screen sharing wasn’t looking very good.
My company self-hosts an instance and it's excellent.
While we’ve had rnnoise integration for a while it was for “noisy environment” notifications, this is the first time we use it to actually filter audio. Also audio worklets weren’t a thing when we first introduced it. I’m not aware of any other open source (and better) models, but if any come up, we’ll certainly check them out!
If you have any involvement with Jigasi or might be in the know -- are there plans to use whisper, for instance, instead of Google's API for transcription? If I recall correctly jigasi is using google's API, local transcription aligns well with the rest of Jitsi's missions.
We do have VOSK support already. I haven’t heard of whisper, but it does sound like a good GSoC project for next year!
If you have any involvement with Jigasi or might be in the know -- are there plans to use whisper, for instance, instead of Google's API for transcription? If I recall correctly jigasi is using google's API, local transcription aligns well with the rest of Jitsi's missions.
We do have VOSK support already. I haven’t heard of whisper, but it does sound like a good GSoC project for next year!
If I have time I'll try to help you guys out. I'm a big fan of what you're doing. :)
Is this a troll comment? Yes, wasm works based on a compiled binary, just like any other program written in a compiled language in the past 50 years. You try to suggest that everyday users of the web are just going into the js sources of webpages and understand whats going on. With the plethora of libraries, frameworks and static optimization used in todays websites, normal people can't really dissect the inner worki…
Sure “everyday users” aren’t clicking “View Source”, but that’s not really what the issue is about. When I was a kid, every piece of software I used was pre-compiled, and therefore opaque. This made it difficult for me to figure out how people made certain things, and after a while I lost interest in programming. When I got back into it later, one thing that made a huge difference was being able to see how various co…
On the other hand, it's never been as easy to contribute to OSS projects as it is now. Github has severely lowered the requirements compared to earlier settings where you had to get an e-mail client, configure it in just the right way, etc. You have live coding youtubers, there are discord communities for all types of technology, and knowledge about programming and technology is extremely available through Google, way more than it was 20 years ago. I think young people still have tons of opportunities to start out.
Bummer, reading the title I thought Jitsi had a new de-noiser because they had RRNoise for some time. Unfortunately RRNoise has not received much advancement for a couple years. It's by now half a decade old tech. I've worked with the WASM version in the past but it can be hit or miss. Sometimes it makes the audio you want a bit weird. It also added something like 10% CPU usage and in the end we disabled it again. I'…
Bummer, reading the title I thought Jitsi had a new de-noiser because they had RRNoise for some time. Unfortunately RRNoise has not received much advancement for a couple years. It's by now half a decade old tech. I've worked with the WASM version in the past but it can be hit or miss. Sometimes it makes the audio you want a bit weird. It also added something like 10% CPU usage and in the end we disabled it again. I'…
While we’ve had rnnoise integration for a while it was for “noisy environment” notifications, this is the first time we use it to actually filter audio. Also audio worklets weren’t a thing when we first introduced it. I’m not aware of any other open source (and better) models, but if any come up, we’ll certainly check them out!
Thanks for the clarification. We also experimented with audio worklets + rrnoise about 1.5 years or so ago but had very mixed results. The potential upside with processing in another thread is clear but some browser and OS combinations just didn't work well and resulted in micro stutters in the audio. I remember Chromium on Linux for example being finicky. Some browsers worked better with smaller buffers, some needed bigger ones. We spent too much time debugging and tuning for different systems and the audio quality improvement was not deemed good enough so we shelved the effort. I guess audio worklets improved since then and probably is more useable by now. Do you guys have some kind of performance monitoring for the noise cancellation or audio in general?
At the time I also spent a few days looking for something better but didn't really find anything. Unfortunately RRNoise is the best we have :( The only other noise cancellation software that actually impressed me was the one from Nvidia but that's not something that one could integrate via WASM and of course wouldn't work on most devices anyways.
Oh what a day it will be where we have energy efficient hardware encoders for AV1 in every device plus some really good noise cancellation. Oh and then we just need internet connections without packetloss :P
Is this a troll comment? Yes, wasm works based on a compiled binary, just like any other program written in a compiled language in the past 50 years. You try to suggest that everyday users of the web are just going into the js sources of webpages and understand whats going on. With the plethora of libraries, frameworks and static optimization used in todays websites, normal people can't really dissect the inner worki…
Sure “everyday users” aren’t clicking “View Source”, but that’s not really what the issue is about. When I was a kid, every piece of software I used was pre-compiled, and therefore opaque. This made it difficult for me to figure out how people made certain things, and after a while I lost interest in programming. When I got back into it later, one thing that made a huge difference was being able to see how various co…
Today's JavaScript "View source" is 90% useless because of Webpack et al. The original program is effectively compiled into obscure and obfuscated lowest-common-denominator JS.
Sorry, but at least in my smartphone, I can understand better the unprocessed audio showcased down in the Web page, than the noise-suppresed audio. How is that?
The original audio is significantly easier to understand. This may be technically interesting, but the noise suppression is aggressive to the point that it's eating critical signal with the noise.
This is the default for online conferencing. Everyone is way better off asking other party to repeat couple of words than listening for all that noise during the whole call.
While we’ve had rnnoise integration for a while it was for “noisy environment” notifications, this is the first time we use it to actually filter audio. Also audio worklets weren’t a thing when we first introduced it. I’m not aware of any other open source (and better) models, but if any come up, we’ll certainly check them out!
If you have any involvement with Jigasi or might be in the know -- are there plans to use whisper, for instance, instead of Google's API for transcription? If I recall correctly jigasi is using google's API, local transcription aligns well with the rest of Jitsi's missions.
The problem for Jigasi's speech-to-text feature with Whisper - or any recent SOTA speech-to-text neural networks, is that they are transformer-based. One of the key features of transformers is that they are very good at processing a sequence with the attention mechanism. But attention inherently needs to see the whole input sequence. So it's difficult to adapt these architectures to perform well in real-time scenarios like captioning meetings.