Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Voice2json: Offline speech and intent recognition on Linux

voice2json.org

Post reply on HN
Print view View original

21–30 of 114 posts

Re: Voice2json: Offline speech and intent recognition on Linux

#21

Post by tootie »

sodality2 wrote: post #6
marcodiego wrote: post #4

Good FLOSS speech recognition and TTS is badly needed. Such interaction should not be left to an oligoply with bad history of not respecting users freedoms and privacy.

Mozilla CommonVoice is definitely trying. I always do a few validations and a few clips if I have a few minutes to spare, and I recommend everyone does. They need volunteers to validate and upload speech clips to create a dataset. https://commonvoice.mozilla.org/en

If you read the doc, it says voice2json is layer on top of the actual voice recognition engine. And it supports mozilla deep speech, pocket sphinx and a few others as the underlying engine.

Re: Voice2json: Offline speech and intent recognition on Linux

#22

Post by sodality2 »

wongarsu wrote: post #19
jfarina wrote: post #11

Earlier quoted context omitted.

I wonder if they use movies and tv; recordings where the script is already available.

That's fine for training your own model, but I don't think you could distribute the training set. That seems like a clear copyright violation, against one of the groups that cares most about copyright. Maybe you could convince a couple of indie creators or state-run programs to licence their audio? But I'm not sure if negotiating that is more efficient than just recording a bit more audio, or promoting the project to…

Does using copyrighted works to train a machine learning model make that model infringing?

Re: Voice2json: Offline speech and intent recognition on Linux

#23

Post by sodality2 »

londons_explore wrote: post #10
marcodiego wrote: post #4

Good FLOSS speech recognition and TTS is badly needed. Such interaction should not be left to an oligoply with bad history of not respecting users freedoms and privacy.

Good speech recognition generally requites massive mountains of training data, both labelled and unlabelled. Massive mountains of data tends to be incompatible with opensource projects. Even Mozilla collecting user statistics is pretty controversial. Imagine someone like Mozilla trying to collect hundreds of voice clips from each of tens of millions of users!!

> Imagine someone like Mozilla trying to collect hundreds of voice clips from each of tens of millions of users!!

They do, and it's working! https://commonvoice.mozilla.org/en

Re: Voice2json: Offline speech and intent recognition on Linux

#24

Post by akiselev »

wongarsu wrote: post #19
jfarina wrote: post #11

Earlier quoted context omitted.

I wonder if they use movies and tv; recordings where the script is already available.

That's fine for training your own model, but I don't think you could distribute the training set. That seems like a clear copyright violation, against one of the groups that cares most about copyright. Maybe you could convince a couple of indie creators or state-run programs to licence their audio? But I'm not sure if negotiating that is more efficient than just recording a bit more audio, or promoting the project to…

It would likely be a lot easier for someone from within the BBC, CBC, PBS, or another public broadcaster to convince their employer to contribute to the models. These organizations often have accessibility mandates with real teeth and real costs implementing that mandate. The work of closed captioning, for example, can realistically be improved by excellent open source speech recognition and TTS models without handing all of the power over to Youtube and the like.

It would still be an uphill battle to convince them to hand over the training set but the legal department can likely be convinced if the data set they contribute back is heavily chopped up audio of the original content, especially if they have the originals before mixing. I imagine short audio files without any of the music, sound effects, or visual content are pretty much worthless as far as IP goes.

Re: Voice2json: Offline speech and intent recognition on Linux

#25

Post by dec0dedab0de »

wongarsu wrote: post #19
jfarina wrote: post #11

Earlier quoted context omitted.

I wonder if they use movies and tv; recordings where the script is already available.

That's fine for training your own model, but I don't think you could distribute the training set. That seems like a clear copyright violation, against one of the groups that cares most about copyright. Maybe you could convince a couple of indie creators or state-run programs to licence their audio? But I'm not sure if negotiating that is more efficient than just recording a bit more audio, or promoting the project to…

That's fine for training your own model, but I don't think you could distribute the training set. That seems like a clear copyright violation, against one of the groups that cares most about copyright.

I'm not sure that is a clear copyright violation. Sure, at a glance it seems like a derivative work, but it may be altered enough that it is not. I believe that collages, and reference guides like cliff notes are both legal.

I think a bigger problem would be that the scripts, and even the closed captioning, rarely match the recorded audio 100%

Re: Voice2json: Offline speech and intent recognition on Linux

#26

Post by marcodiego »

sodality2 wrote: post #22
wongarsu wrote: post #19

Earlier quoted context omitted.

That's fine for training your own model, but I don't think you could distribute the training set. That seems like a clear copyright violation, against one of the groups that cares most about copyright. Maybe you could convince a couple of indie creators or state-run programs to licence their audio? But I'm not sure if negotiating that is more efficient than just recording a bit more audio, or promoting the project to…

Does using copyrighted works to train a machine learning model make that model infringing?

GP is not talking about the model but about the training data set.

Re: Voice2json: Offline speech and intent recognition on Linux

#27

Post by sodality2 »

marcodiego wrote: post #26
sodality2 wrote: post #22

Earlier quoted context omitted.

Does using copyrighted works to train a machine learning model make that model infringing?

GP is not talking about the model but about the training data set.

I am aware, I'm asking if the model, however, is infringing. Surely you can't distribute them in a dataset but is training on copyrighted data legal, and can you distribute that model?

Re: Voice2json: Offline speech and intent recognition on Linux

#28

Post by teraflop »

sodality2 wrote: post #6
marcodiego wrote: post #4

Good FLOSS speech recognition and TTS is badly needed. Such interaction should not be left to an oligoply with bad history of not respecting users freedoms and privacy.

Mozilla CommonVoice is definitely trying. I always do a few validations and a few clips if I have a few minutes to spare, and I recommend everyone does. They need volunteers to validate and upload speech clips to create a dataset. https://commonvoice.mozilla.org/en

I like the idea, and decided to try doing some validation. The first thing I noticed is that it asks me to make a yes-or-no judgment of whether the sentence was spoken "accurately", but nowhere on the site is it explained what "accurate" means, or how strict I should be.

(The first clip I got was spoken more or less correctly, but a couple of words are slurred together and the prosody is awkward. Without having a good idea of the standards and goals of the project, I have no idea whether including this clip would make the overall dataset better or worse. My gut feeling is that it's good for training recognition, and bad for training synthesis.)

This seems to me like a major issue, since it should take a relatively small amount of effort to write up a list of guidelines, and it would be hugely beneficial to establish those guidelines before asking a lot of volunteers to donate their time. I don't find it encouraging that this has been an open issue for four years, with apparently no action except a bunch of bikeshedding: https://github.com/common-voice/common-voice/issues/273

Re: Voice2json: Offline speech and intent recognition on Linux

#29

Post by timvisee »

londons_explore wrote: post #10
marcodiego wrote: post #4

Good FLOSS speech recognition and TTS is badly needed. Such interaction should not be left to an oligoply with bad history of not respecting users freedoms and privacy.

Good speech recognition generally requites massive mountains of training data, both labelled and unlabelled. Massive mountains of data tends to be incompatible with opensource projects. Even Mozilla collecting user statistics is pretty controversial. Imagine someone like Mozilla trying to collect hundreds of voice clips from each of tens of millions of users!!

Another problem is that the models tend to get very very large for what I've seen. A gigabyte to 10s of gigabyes is an undesirable requirement on your local machine.

Re: Voice2json: Offline speech and intent recognition on Linux

#30

Post by bmn__ »

Has anyone had any success getting the software to work?

It's entirely unpackaged: https://repology.org/projects/?search=voice2json https://pkgs.org/search/?q=voice2json

Docker image is broken, how'd that happen?

    $ voice2json --debug train-profile
    ImportError: numpy.core.multiarray failed to import
    Traceback (most recent call last):
      File "/usr/lib/voice2json/.venv/lib/python3.7/site-packages/deepspeech/impl.py", line 14, in swig_import_helper
        return importlib.import_module(mname)
      File "/usr/lib/python3.7/importlib/__init__.py", line 127, in import_module
        return _bootstrap._gcd_import(name[level:], package, level)
      File "", line 1006, in _gcd_import
      File "", line 983, in _find_and_load
      File "", line 967, in _find_and_load_unlocked
      File "", line 670, in _load_unlocked
      File "", line 583, in module_from_spec
      File "", line 1043, in create_module
      File "", line 219, in _call_with_frames_removed
    ImportError: numpy.core.multiarray failed to import
Post reply on HN