Live data from Hacker News

Whisper – open source speech recognition by OpenAI

openai.com

391–400 of 508 posts

Re: Whisper – open source speech recognition by OpenAI

#391

Earlier quoted context omitted.

It's a micromachines ad from the '80s. He talked like that in all of them! As for speed, to a computer we don't talk very fast, not even that guy. I wonder if it could handle Rap God by Eminem....Let's find out!

Did you find out :D?

I did! There are a few places it transcribes incorrectly, but overall I'm very impressed. Here's the first ~30 seconds:

    [00:00.000 --> 00:09.000]  Look, I was going to go easy on you, not to hurt your feelings, but I'm only going to get this one chance.
    [00:09.000 --> 00:11.000]  Something's wrong, I can feel it.
    [00:11.000 --> 00:17.000]  It's just a feeling I've got, like something's about to happen, but I don't know what.
    [00:17.000 --> 00:21.000]  If that means what I think it means, we're in trouble, big trouble.
    [00:21.000 --> 00:24.000]  Had to be as bananas as you say, I'm not taking any chances.
    [00:24.000 --> 00:26.000]  You're just one to die for.
    [00:26.000 --> 00:32.000]  I'm beginning to feel like a rap god, rap god. All my people from the front to the back nod, back nod.

Re: Whisper – open source speech recognition by OpenAI

#392
I tried it out on a Hindi speech (https://www.youtube.com/watch?v=4EpfJxKyosE). The transcription starts off decent, but kind of gets stuck repeating the same thing at the 02:40 mark:

    [00:00.000 --> 00:10.000]  पचास ताल में हमने प्रगती किये, इससे को इंटार नहीं कर सकता।
    [00:10.000 --> 00:20.000]  छुनाओ के दौरान वोट मांगते हुए, सरकार की नीतियों पर कठोर से कठोर प्रहार करते हुए,
    [00:20.000 --> 00:28.000]  और पुरानी सरकार की नीतियों नहीं आलोचना करने के लिए लैक बहुत सामग्री थी।
    [00:28.000 --> 00:35.000]  हर जगे मैंने ये कहा कि मैं उन लोगों में से नहीं हूँ, जो पचास वर्च की उपलड्यों पर पानी फिर दे।
    [00:35.000 --> 00:43.000]  ऐसा करना देश के पुर्षार्थ पर पानी फिरना होगा। ऐसा करना देश के किसान के साथ अन्याय करना होगा।
    [00:43.000 --> 01:01.000]  मल्दूर के साथ जात्ती करनी होगा। आम आद्मी के साथ भी वो अच्छा व्योहार नहीं होगा। जो स्वाल आज मन में उच्छा है और उच्छना चाही है। आदावी को पचास साथ होने आये, हम जैनती मनाने जा रहे हैं।
    [01:01.000 --> 01:18.000]  आज देश की स्तिती क्या है। हम पिछर के होगे हैं। प्रगती की दोड़ में, जो देश हमारे साथ आजाद हुए थे, वो हम से आगे बढ़ के। जो देश हमारे बाच जन में थे, वो हमें पीचे छोड़ थे।
    [01:18.000 --> 01:34.000]  दुनिया के गरी तम देशों में हमारी गड़न आये। वीस फीज़ी से जाना लो गरीबी की रेका के नीचे। राक्तपती महुदाय के विभाशन में गाऊं का उल्लेक हैं ना पीरे का पानी नहीं।
    [01:34.000 --> 01:50.000]  हम प्राथमी शिक्षा अनिवारे नहीं कर सकते हैं। लड्कियों की शिक्षा की उपेक्षा हो रही हैं। लड्कि का जन्म लेना तो इस देश में अभी तक एक अभिशाप है।
    [01:50.000 --> 02:07.000]  क्या सरकारी कदम उठाकर समाज में जाग्दृती पैदा करकें। क्या सब लोगों को जुटाकर ये तो ऐसा काम है जिस में कोई दलबंदी के लिए इस्थान नहीं। हम देश का नक्षा नहीं बदल सकते हैं। देश में साधनों की कमी नहीं है।
    [02:07.000 --> 02:07.000]  और साधनों की अगर कमी है तो उसको ठीक दन्त से प्राप्त किया जा सकता है। साधन बड़ाए भी जा सकते है। लेकिन जो साधन हैं उनका ठीक उपयोग नहीं हो रहा। जंता के उपर टेक्स लगाकर जो दन्नि कप्ता किया जाता है। उसका लाग जंता तक नहीं पहु
    [02:37.000 --> 02:37.000]  रख्कम जाती है। विदेशी बैंको में दन जाने का सिल्सिला अभी तक क्यों काएं है। उसको लोकने के लिए क्या कदम उठाएगे। हम विदेशी पूजी के लिए प्रैत्रशील हैं विदेशी पूजी आए और अगर विदेशी पूजी आती है अच्छे दन्त की टेक
    [03:07.000 --> 03:07.000]  अच्छे दन्त की पूजी आती है अच्छे दन्त की पूजी आती है अच्छे दन्त की पूजी आती है अच्छे दन्त की पूजी आती है
    [03:37.000 --> 03:39.000]  अच्छे दन्त की पूजी आती है अच्छे दन्त की पूजी आती है
    [04:07.000 --> 04:09.000]  अच्छे दन्त की पूजी आती है अच्छे दन्त की पूजी आती है
    [04:37.000 --> 04:39.000]  अच्छे दन्त की पूजी आती है अच्छे दन्त की पूजी आती है
The translation does a much better job however:

    [00:00.000 --> 00:10.000]  In the last 50 years, we have made progress, no one can deny this.
    [00:10.000 --> 00:20.000]  During the elections, while asking for votes, while attacking the government's policies harshly,
    [00:20.000 --> 00:28.000]  and to criticize the policies of the old government, a lot of material was needed.
    [00:28.000 --> 00:35.000]  Everywhere, I have said that I am not one of those people who pour water on the fruits of 50 years.
    [00:35.000 --> 00:39.000]  To do this, we will have to pour water on the efforts of the country.
    [00:39.000 --> 00:43.000]  To do this, we will have to do injustice with the farmers of the country.
    [00:43.000 --> 00:45.000]  We will have to do caste with the laborers.
    [00:45.000 --> 00:50.000]  Even with the common man, that will not be a good behavior.
    [00:50.000 --> 00:55.000]  The question that arises in the mind today and should arise,
    [00:55.000 --> 01:01.000]  Freedom has come to be 50 years, we are going to celebrate.
    [01:01.000 --> 01:04.000]  What is the situation of the country today?
    [01:04.000 --> 01:07.000]  Why did we get separated?
    [01:07.000 --> 01:14.000]  In the race of progress, the country that got freedom along with us, they went ahead of us.
    [01:14.000 --> 01:19.000]  The country that was after us, they left us behind.
    [01:19.000 --> 01:25.000]  In the poorest countries of the world, they counted us.
    [01:25.000 --> 01:29.000]  20% of the population is below the poverty line.
    [01:29.000 --> 01:35.000]  In the speech of the President, there is no mention of villages or drinking water.
    [01:35.000 --> 01:39.000]  We cannot enforce primary education.
    [01:39.000 --> 01:43.000]  The education of girls is being neglected.
    [01:43.000 --> 01:50.000]  The birth of a girl is still a curse in this country.
    [01:50.000 --> 01:55.000]  Is it by taking government steps, by creating awareness in the society?
    [01:55.000 --> 02:01.000]  Is it by uniting all the people that there is no place for party?
    [02:01.000 --> 02:05.000]  Can't we change the map of the country?
    [02:05.000 --> 02:08.000]  There is no shortage of resources in the country.
    [02:08.000 --> 02:14.000]  And if there is a shortage of resources, it can be obtained in the right way, resources can be increased.
    [02:14.000 --> 02:21.000]  But the resources that are there, they are not being used properly.
    [02:21.000 --> 02:30.000]  The wealth that is collected by taxing the public, its profit does not reach the public, it does not reach the common man.
    [02:30.000 --> 02:32.000]  Where does it go?
    [02:32.000 --> 02:35.000]  Whose pockets are filled?
    [02:35.000 --> 02:39.000]  Whose treasury does that money go to?
    [02:39.000 --> 02:44.000]  Why is the chain of money going to foreign banks still established?
    [02:44.000 --> 02:47.000]  What steps have been taken to stop it?
    [02:47.000 --> 02:52.000]  We are motivated for foreign worship, foreign worship has come.
    [02:52.000 --> 03:01.000]  And if foreign worship comes for good technology, for infrastructure,
    [03:01.000 --> 03:06.000]  for education, then no one will object.
    [03:06.000 --> 03:11.000]  I believe that our communist friends will not object either.
    [03:11.000 --> 03:19.000]  But is the maximum use of the resources in the country happening?
    [03:19.000 --> 03:26.000]  Is it not true that corruption has become a national disease?
    [03:26.000 --> 03:31.000]  I remember that Swargi Rajiv Gandhi had said in a speech that I send one rupee from Delhi,
    [03:31.000 --> 03:36.000]  but where I send the rupee, as I reach there, 19 paise are left.
    [03:36.000 --> 03:41.000]  I asked him how this miracle happens.
    [03:41.000 --> 03:47.000]  Bhaskar said that when the rupee runs, it shrinks.
    [03:47.000 --> 03:54.000]  The rupee shrinks, it gets into the hand, it goes into the pocket, it becomes small.
    [03:54.000 --> 03:58.000]  It is difficult to recognize the rupee.
    [03:58.000 --> 04:02.000]  The rupee can be hidden.
    [04:02.000 --> 04:06.000]  The situation of the currency of the country is not good.
    [04:06.000 --> 04:10.000]  First, the government expenditure has increased, it is increasing.
    [04:10.000 --> 04:17.000]  It needs common consent to reduce without reducing.
    [04:17.000 --> 04:24.000]  No one can work in the same way.
    [04:24.000 --> 04:27.000]  Yes, our old Prime Minister Narasimha Raoji,
    [04:27.000 --> 04:34.000]  if he would have tried in this direction after stabilizing himself, then he would have succeeded.
    [04:34.000 --> 04:47.000]  But he was stuck in some such things that he could not pay attention to these problems.

Re: Whisper – open source speech recognition by OpenAI

#393
post #347
post #61

Earlier quoted context omitted.

Little details. The second sentence is really bizarre: > Nous établissons que l'utilisation de données d'un tel nombre et d'une telle diversité est la raison pour laquelle le système est à même de comprendre de nombreux accents... It doesn't sound natural at all. An idiomatic formulation would be more along the lines of: Le recours à un corpus [de données] si riche et varié est ce qui permet au système de comprendre…

That's interesting, as a québécois I don't agree with any of this. The only thing that raised an eyebrow was "est à même de", but if turns out it's just another way of saying "capable de", I guess it's simply not a common idiom around here. Aside from that, I found the wording flowed well even if I personally would've phrased it differently.

Mistery solved. It was a quebecois

Re: Whisper – open source speech recognition by OpenAI

#394

Earlier quoted context omitted.

How do you get this to translate instead of just transcribe?

Just specify language and record an audio in another language. >result = model.transcribe("audio.wav", language="english")

That actually seems to set the language for it to transcribe (as opposed to it guessing), with the following triggering a translation to English:

result = model.transcribe("audio.wav", task="translate")

But your post helped me figure out the above, so thank you!

Re: Whisper – open source speech recognition by OpenAI

#395
post #242

Their Scottish accent example is pretty good, I'd like to see it work on some very strong English accents like this one: https://www.youtube.com/watch?v=nJ7QB3om-QY

Detected language: english [00:00.000 --> 00:05.400] Gordy and County Kerry are investigating the theft of up to 60 sheep on Mount Brandon. [00:05.400 --> 00:10.400] One of the farmers is offering a reward for information leading to the return of the use, [00:10.400 --> 00:12.200] which are worth thousands of euro. [00:12.200 --> 00:14.200] Well, I'm fine with that. [00:14.200 --> 00:15.200] That's right. [00:15.200…

Wow!

Re: Whisper – open source speech recognition by OpenAI

#396
post #366

I tried running it in realtime with live audio input (kind of). If you want to give it a shot, you can find the python script in this repo: https://github.com/tobiashuttinger/openai-whisper-realtime A bit more context on how it works: The systems default audio input is captured with python, split into small chunks and is then fed to OpenAI's original transcription function. It tries (currently rather poorly) to detec…

Quoted post unavailable.

impressive

Re: Whisper – open source speech recognition by OpenAI

#399
post #68
post #35

Be wary of using this model - the licensing of this model seems sketchy. Several of the datasets used for training like WSJ and TED-LIUM have clear non-commercial clauses. I'm not a lawyer but releasing a model as "MIT" seems dubious, and hopefully OpenAI has paid for the appropriate licenses during training as they are no longer a research-only non profit.

This is a big dispute right now: OpenAI and other AI companies generally take the position that models learning from data does not make the output of the models a derivative work of that data. For example, GitHub Co-pilot uses all publicly available GitHub code regardless of license, and DALLE-2/StableDiffusion/etc use lots of non-free images. I don't think this has been challenged in court yet, and I'm very curious…

> models learning from data does not make the output of the models a derivative work of that data

Most of the debate seems to be happening on the question of whether everything produced by models trained on copyrighted work represents a derivative work. I argue that at the very least some of it does; so the claim said to be made by the AI companies (see quote above) is clearly a false one.

We're in a weird place now where AI is able to generate "near verbatim" work in a lot of cases, but I don't see an obvious case for treating this any differently than a human reproducing IP with slight modifications. (I am not a lawyer.)

For example, copyright law currently prevents you from selling a T-shirt with the character Spider-Man on it. But plenty of AI models can give you excellent depictions of Spider-Man that you could put on a T-shirt and try to sell. It's quite silly to think that any judge is going to take you seriously when you argue that your model, which was trained on a dataset that included pictures of Spider-Man, and was then asked to output images using "Spider-Man" as a search term, has magically circumvented copyright law.

(I think there's a valid question about whether models represent "derivative work" in the GPL sense specifically, but I'm using the idea more generally here.)

Re: Whisper – open source speech recognition by OpenAI

#400

A notebook is available to try with your microphone on Colab here: https://colab.research.google.com/drive/1nBZ-pDIaIi3N1DIIXvJ... I'm surprised by the quality on non-English languages, given that 80+% of the training data is English, and the rest is split between tens of languages.

How do you get this to translate instead of just transcribe?

To be more specific than the above:

1. Make sure you're using a model that isn't suffixed with `.en` (`base`, not `base.en). 2. Use `model.transcribe(your_input_audio, language='Japanese', task='translate')` ... with the appropriate input language.

Post reply on HN