Live data from Hacker News

TSAC: Low Bitrate Audio Compression

bellard.org

51–60 of 101 posts

Re: TSAC: Low Bitrate Audio Compression

#51
Bellard strikes again...

Are we almost converting music to MIDI at this point?

As I understand it the model is learning the landscape of sound combinations that are interesting to humans and as such there will be no combination of raw bytes in the recorded file that will result in white noise (for example) being heard because this is never trained for.

What if it was though?

Re: TSAC: Low Bitrate Audio Compression

#52

This doesn't have much of a use case. - Can't use it in telephony (obvious application for low bitrates); phone handsets and headsets don't have the power to do it in real time. - Very small files of good quality would be useful in tiny embedded systems that have low flash space: but what systems of that type have the processing power for decoding? Very low storage more or less goes hand in hand with weak processing.…

There's plenty of use cases, just because you can't think of them doesn't mean there aren't any. Neural network based audio codecs like TSAC, Descript, Encodec, Soundstream are used for music and speech generation, audio upsampling, pre-training acoustic language models, speech-to-speech translation etc.

Check out the citations of Encodec (Facebook's open sourced audio codec) for more examples: https://scholar.google.com/scholar?cites=1126914113099467682...

Re: TSAC: Low Bitrate Audio Compression

#53

This doesn't have much of a use case. - Can't use it in telephony (obvious application for low bitrates); phone handsets and headsets don't have the power to do it in real time. - Very small files of good quality would be useful in tiny embedded systems that have low flash space: but what systems of that type have the processing power for decoding? Very low storage more or less goes hand in hand with weak processing.…

Am I missing something in thinking that this could be alleviated like every other compression algorithm by implementing it via a hardware codec?

It's a neural network, not a traditional compression algorithm. It would be difficult to implement this efficiently in an ASIC AFAIK, but if there are any hardware designers that disagree please chime in.

Re: TSAC: Low Bitrate Audio Compression

#54
One of the DAC authors here (the codec that this builds off of). Very cool work! Would love to see some more detail on the modifications to DAC. Boosting the capacity with a transformer makes sense to me.

Makes me happy to see DAC getting built on! Thanks!

Re: TSAC: Low Bitrate Audio Compression

#55
post #13

Always love a good bellard ship. This is definitely better than some of the others out there. I threw together some comparisons here at 7kb/s for mp3/opus/aac: https://non.io/TSAC-Comparisons Happy to add other comparisons if others want any. Overall, it's FAR better at these lower bit rates, but that doesn't mean it's necessarily good. One issue I see off the bat is that volume is fairly inconsistent in the output f…

Thats much more helpful comparison. TSAC clearly works far better than mp3 and opus for low bitrates.

Re: TSAC: Low Bitrate Audio Compression

#56

This doesn't have much of a use case. - Can't use it in telephony (obvious application for low bitrates); phone handsets and headsets don't have the power to do it in real time. - Very small files of good quality would be useful in tiny embedded systems that have low flash space: but what systems of that type have the processing power for decoding? Very low storage more or less goes hand in hand with weak processing.…

> Can't use it in telephony (obvious application for low bitrates); phone handsets and headsets don't have the power to do it in real time.

Not now but in another 5 years they will start to and in 10 years all new ones will probably have the power for this. I find it really exciting although it will consume more battery to run that; but if that is less than the radio antenna requires then it might make sense.

Re: TSAC: Low Bitrate Audio Compression

#57
post #53

Earlier quoted context omitted.

Am I missing something in thinking that this could be alleviated like every other compression algorithm by implementing it via a hardware codec?

It's a neural network, not a traditional compression algorithm. It would be difficult to implement this efficiently in an ASIC AFAIK, but if there are any hardware designers that disagree please chime in.

I figured some sort of NPU + tuned hardware would be enough, but I'm just going off of intuition.

Re: TSAC: Low Bitrate Audio Compression

#58
This appears to sit just in the middle of something that could be used for music, with a higher bit rate, still much lower than other competing codecs, and something very effective for voice communication, but shrinking the bandwidth (thus the bit rate) also to limit artifacts. Not an expert in the field, anyway I think the supplied examples aren't the best ones to show its potential.

Re: TSAC: Low Bitrate Audio Compression

#60

This doesn't have much of a use case. - Can't use it in telephony (obvious application for low bitrates); phone handsets and headsets don't have the power to do it in real time. - Very small files of good quality would be useful in tiny embedded systems that have low flash space: but what systems of that type have the processing power for decoding? Very low storage more or less goes hand in hand with weak processing.…

Archiving over a long period of time might be a use case.

I often wonder, how much of the data currently in circulation will be lost at some point? HDD/SSD last a couple of years. Most of the data in the cloud will be copied over, but some will be lost. If you extrapolate to a 1000, 1 000 000 years, how much will remain? Will something survive the civilization collapse? I guess most people don't care, but some will ...

One way to make data mediums last longer is to make them lower density, and for that such super-low bitrate could be useful.

Post reply on HN