Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Opus 1.5 released: Opus gets a machine learning upgrade

opus-codec.org

Post reply on HN
Print view View original

131–140 of 151 posts

Re: Opus 1.5 released: Opus gets a machine learning upgrade

#131

Post by spacechild1 »

RossBencina wrote: post #125
spacechild1 wrote: post #24

I'm using Opus as one of the main codecs in my peer-to-peer audio streaming library ( https://git.iem.at/cm/aoo/ - still alpha), so this is very exciting news! I'll definitely play around with these new ML features!

> peer-to-peer audio streaming library Interesting :)

Ha, now that is what I'd call a suprise! The "group" concept has obviously been influenced by oscgroups. And of course I'm using oscpack :)

AOO has already been used successfully in a few art projects. It's also used under hood by SonoBus. The Pd objects are already stable, I hope to finish the C/C++ API and add some code examples soon.

Any feedback from your side would of course be very appreciated.

Re: Opus 1.5 released: Opus gets a machine learning upgrade

#132

Post by frumiousirc »

How about adding a text "subtitle" stream to the mix. The encoder may use ML to perform speech-to-text. The decoder may then use the text, along with the audio surrounding the audio drop outs, to feed a conditional text-to-speech DNN. This way the network does not have to learn the harder problem of blindly interpolating across the drop outs from just the audio. The text stream is low bitrate so it may have substantial redundancy in order to increase the likelihood that any given (text) message is received.

Re: Opus 1.5 released: Opus gets a machine learning upgrade

#133

Post by out_of_protocol »

maep wrote: post #128
out_of_protocol wrote: post #15

Earlier quoted context omitted.

They chose to make totally new inferior LC3 codec though. Also, on my system (Android phone + BTR5/BTR15 Bluetooth DAC + Sennheiser H600) all options sound realy crappy compared to plain old usb, everything else is the same. LDAC 990kbps is less crappy, by sheer brute force. I suspect it's not only codec but other co-factors as well (like mandatory DSP on phone side)

"Inferior" is relative. The main focus of LC3 was, as the name suggests, complexity. This is hearsay: Bluetooth SIG considered Opus but rejected it because it was computationally too expensive. This came out of the hearing aid group, where battery life and complexity are a major restriction. So when you compare codecs in this space, the metric you want to look at is quality vs. CPU cycles. In that regard LC3 outperfo…

Regarding complexity, do you have any hard numbers? Can't find anything more than handwaving

Re: Opus 1.5 released: Opus gets a machine learning upgrade

#134

Post by aredox »

jmvalin wrote: post #91
aredox wrote: post #82

>That's why most codecs have packet loss concealment (PLC) that can fill in for missing packets with plausible audio that just extrapolates what was being said and avoids leaving a hole in the audio ...How far can ML PLC "hallucinate" audio? A sound , a syllable, a whole word, half a sentence? Can I trust anymore what I hear?

What the PLC does is (vaguely) equivalent to momentarily freezing the image rather than showing a blank screen when packets are lost. If you're in the middle of a vowel, it'll continue the vowel (trying to follow the right energy) for about 100 ms before fading out. It's explicitly designed not to make up anything you didn't say -- for obvious reasons.

Reassuring - thanks for clarifying that up.

Re: Opus 1.5 released: Opus gets a machine learning upgrade

#135

Post by m3kw9 »

samus wrote: post #122
m3kw9 wrote: post #94

Some people hyping it as AGI on social media

Sadly, I see it even on forums where one might think people have background in technology...

The tech background is likely IT and not AI. They used ChatGPT and they thought it’s conscious

Re: Opus 1.5 released: Opus gets a machine learning upgrade

#136

Post by thomastjeffery »

samus wrote: post #114
thomastjeffery wrote: post #58

Earlier quoted context omitted.

Usually regular algorithms aren't generating data that pretends to be raw data. That's the significant difference here.

Not really. Any lossy codec is generating data that pretends to be close to the raw data.

Yes, but the holes were intentionally constructed such that the end result is predictable.

There is a difference between pretending to be the original raw data, and pretending to be whatever data will most likely fit.

Re: Opus 1.5 released: Opus gets a machine learning upgrade

#137

Post by shwaj »

viraptor wrote: post #105
shwaj wrote: post #100

Earlier quoted context omitted.

I don't think it's a given that we shouldn't keep using that codec. For example, maybe the improvement is due to an open source hacker working in their spare time to make the world a better place. Do we tell them their contribution isn't welcome until it meets the community's benchmark for equity? Your same argument can also be used to degrade the performance for all other groups, so that group X isn't unfairly disad…

A person creating a codec doesn't choose if it's globally adopted. System implementors (like for example Slack) do. You're don't have to tell the open source dev anything. You don't owe them to include their implementation. And if their contribution was to the final system, sure, it's the owner's choice what the threshold for acceptable contribution is. In the same way they can set any other benchmark. > Your same ar…

The grandparent provided an argument why we might not want to use an algorithm, even if it provided a Pareto improvement.

I suggested that the same argument could be used to say that we should actively degrade performance of the algorithm, in the name of equity. This is absurd, and illustrates that the GP argument is maybe not as strong as it appears.

Re: Opus 1.5 released: Opus gets a machine learning upgrade

#138

Post by thomastjeffery »

shwaj wrote: post #97
thomastjeffery wrote: post #58

Earlier quoted context omitted.

Usually regular algorithms aren't generating data that pretends to be raw data. That's the significant difference here.

Can you precisely define what you mean by "generating" and "pretends", in such a way that this neural network does both these things, but a conventional modern audio codec doesn't? "Pretends" is a problematic choice of words, because it anthropomorphizes the algorithm. It would be more accurate and less misleading to replace "pretends to be" with "approximates". But then it wouldn't serve your goal of (seeming to) es…

I was avoiding the word "approximate", because that implies a connection to the original raw data.

A generative model guesses what data should be filled in, based on what is present in its own model. This process is totally ignorant of the original (missing) data.

To contrast, a lossy codec works directly with the original data. It chooses what to throw out based on what the algorithm itself can best reproduce during playback. This is why you should never transcode from one lossy codec to another: the holes will no longer line up with the algorithm's hole-filling expectations.

Re: Opus 1.5 released: Opus gets a machine learning upgrade

#139

Post by jmvalin »

frumiousirc wrote: post #132

How about adding a text "subtitle" stream to the mix. The encoder may use ML to perform speech-to-text. The decoder may then use the text, along with the audio surrounding the audio drop outs, to feed a conditional text-to-speech DNN. This way the network does not have to learn the harder problem of blindly interpolating across the drop outs from just the audio. The text stream is low bitrate so it may have substanti…

Actually, what we're doing from DRED isn't that far from what you're suggesting. The difference is that we keep more information about the voice/intonation and we don't need the latency that would otherwise be added by an ASR. In the end, the output is still synthesized from higher-level, efficiently compressed information.

Re: Opus 1.5 released: Opus gets a machine learning upgrade

#140

Post by samus »

thomastjeffery wrote: post #136
samus wrote: post #114

Earlier quoted context omitted.

Not really. Any lossy codec is generating data that pretends to be close to the raw data.

Yes, but the holes were intentionally constructed such that the end result is predictable. There is a difference between pretending to be the original raw data, and pretending to be whatever data will most likely fit .

And that's why Packet Loss Concealment is only used to fill in the occasional lost packet. That way, the occasional vowel or the end of a syllable could be bridged over. Other improvements exist to prevent packet loss in the first place that are much less in make-samples-up territory.
Post reply on HN