Speech Recognition Is Not Solved

I would argue that it can't be solved without severe privacy implications. Apple's talk-to-text, for example, should know my son's name by now, but it doesn't. And while I'm mildly frustrated at the fact that I have to go back and edit text messages on a regular basis, I'm pretty glad that Apple doesn't know my son's name. I'd hate for a company like Facebook to have access to all of the proper nouns in my everyday life.

I too would be interested in pointers to the leading open source options.

Just yesterday there was a Show HN built with the https://github.com/kaldi-asr/kaldi project, emscripten-ized: https://news.ycombinator.com/item?id=15534531

> Latency: With latency, I mean the time from when the user is done speaking to when the transcription is complete. ... While this may sound extreme, remember that producing the transcript is usually the first step in a series of expensive computations.

For many applications, making a transcription seems like an unnecessary step and source of errors. Skipping transcription when the user doesn't need it (most cases where I use it) would seem like a way to get some gain, but perhaps at the reduction of debuggability.

> For example in voice search the actual web-scale search has to be done after the speech recognition.

That's an area where literal exact transcription is usually required. But even then, Siri/Cortana/Alexa might be better off trying to figure out the meaning of what someone's asking rather than figuring out the exact words spoken in order to return the best results.

Most people are quite bad at formulating good internet searches without a lot of trial and error. Let Google listen to a person talk about the problem they have or issue and then formulate the best results for that instead of forcing us to come up with the exact right phrasing to get appropriate results. It would help tremendously with the synonym and homophone issues that are so annoying now.

This is, to me, one of the major problems with many algorithmic solutions to problems. An x% increase does in precision, F measure or any other score does in no way mean that the results are better.

I've repeatedly seen improvements to traditional measures that make the subjective result worse.

It's incredibly hard to measure and solve (if anyone has good ideas please let me know). I check a lot of sample data manually when we make changes, doing that (with targeting at important cases) is really the only way I think to do things.

Just the other day, I participated in a discussion about how "language identification" is a solved problem -- in fact, hasn't it been solved for a decade?

As anyone who's had to use langid in practice will testify, it's solved only as long as:

A) you want to identify 1 out of N languages (reality: a text can be in any language, outside your predefined set)

B) you assume the text is in exactly one language (reality: can be 0, can be multiple, as is common with globalized English phrases)

C) you don't need a measure of confidence (most algos give an all-or-nothing confidence score [0])

D) the text isn't too short (twitter), too noisy (repeated sections ala web pages, boilerplate), too regional/dialect, etc.

In other words, not solved at all.

In my experience, the same is true for any other ML task, once you want to use it in practice (as opposed to "write an article about").

The amount of work to get something actually working robustly is still non-trivial. In some respects, things have gotten worse over the past years due to a focus on cranking up the number of model parameters, at the expense of a decent error analysis and model interpretability.

[0] https://twitter.com/RadimRehurek/status/872280794152054784