Great summary! I also recently wrote a post digging into the internals of wav2vec with illustrations:
The Illustrated Wav2vec - https://jonathanbgn.com/2021/06/29/illustrated-wav2vec.html
Wav2vec Overview: Semi and Unsupervised Speech Recognition
21–25 of 25 posts
Re: Wav2vec Overview: Semi and Unsupervised Speech Recognition
#22Does anyone know of some good open sourced projects for OCR? Tesseract always seems to be the default, and then it seems Google cloud, and other services are miles ahead. However, for those who don't want to rely on the big tech companies, are there any comparable alternatives?
I recently came across CRAFT wich appears to have come out of the ICDAR2017 Robust reading challenge. It performed better than expected. I only tested a few images so please don't take my word for it. That led me to PaddleOCR. There is still plenty of room for improvement but I found it way more convenient to use for my purposes than messing with Tesseract. https://github.com/clovaai/CRAFT-pytorch https://github.com/…
Thanks for the suggestions - I've tried Paddle before, however was looking for a local only version - seems like Paddle uses external sources (cn) for some of its OCR features, and unfortunately that won't work in this line of work
Re: Wav2vec Overview: Semi and Unsupervised Speech Recognition
#23One addendum to the linked post's notes: > SoTa in low-resource setting Libri-light by a lot on WER clean test 100h labeled: others ~4 vs theirs ~2.5 > SoTa on high-resource noisy data (3.3 vs 3.4) close to SoTa on clean data This note isn't super specific, but it's outdated if I'm understanding it correctly. To my understanding, the SOTA on this data is held by Conformer 1B (a 1 billion parameter model), at 1.4 clea…
I always wonder how people figure out these successful gigantic models if it takes hundreds of TPUs and days to train them. I recently bought rtx 3090 in hopes of playing around with some computer vision applications but I guess having 24GB VRAM is nothing if I want to get something SOTA working.
So many papers casually mention their hyperparameters, neglecting to mention that those specific numbers are often necessary for performance. Something you don't realize unless you play around with their code…
Re: Wav2vec Overview: Semi and Unsupervised Speech Recognition
#24As someone who's an idiot about machine learning, is it possible to run this code in reverse? e.g. take the generated (or novel) vectors and convert them back into audio/waveforms?
Generalized reverse projection through even non recurrent neural networks is still an open research problem. So no in this case.
I wouldn't rule it out entirely; you could use these as a replacement for linguistic inputs in a tts system, for example, and I imagine it wouldn't be totally terrible. It would still end up being a pretty heavy system, though, with many other parts.
Re: Wav2vec Overview: Semi and Unsupervised Speech Recognition
#25Does anyone know of some good open sourced projects for OCR? Tesseract always seems to be the default, and then it seems Google cloud, and other services are miles ahead. However, for those who don't want to rely on the big tech companies, are there any comparable alternatives?
Did anything eventually happen with Ocropus?