Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Transcribe.cpp

workshop.cjpais.com

Post reply on HN
Print view View original

111–120 of 180 posts

Re: Transcribe.cpp

#111

Post by sipjca »

rcarmo wrote: post #84
sipjca wrote: post #42

Earlier quoted context omitted.

Contributions are always welcome! There’s a WIP diarization PR rn, and after it’s merged would love to have support if it fits well into the interface. And if not would love to figure out a good interface for it

Yeah, diarization is the real feature these days. STT needs uniformization, but quality of diarization is what is setting personal solutions apart in this field.

For sure, it was not initially a target because I didn’t need it for Handy but I do understand the importance in the broader context

Re: Transcribe.cpp

#112

Post by sipjca »

oezi wrote: post #44
sipjca wrote: post #36

Earlier quoted context omitted.

Yep, but I am in the process of also porting NVIDIAs Sortformer for multi speaker diarization as well :) I’m not sure how many specific models will be supported as the library is more focused on transcription specifically. But the models which support diarization natively must be supported I think. And parakeet multitalker was the primary driving force for this change

How close do you aim for when it comes to drop-in vs whisper.cpp? Are timestamps per word and character something aimed for? How about multi-lingual transcription or hallucination suppression? The github page doesn't seem to go into depth on these orthogonal topics. May have missed it.

Eventually I would like to be more fully drop in compatible, right now some feature support is a bit sparse. And whisper has so much work done to it over the years so it’s hard to support every possible thing. Right now it’s a more bog standard implementation than anything special. Right now stabilizing the core header is probably among the primary goal, but if people want to contribute model specific things im happy to review test and pull in. Whisper is a good case for this as there is a header extension already so it’s easier

Re: Transcribe.cpp

#113

Post by verst »

rmunn wrote: post #101

Looks very cool. One thing I have been looking for, which this doesn't seem to cover (at least I didn't see any mention of IPA in the model documentation), is a way to transcribe unknown languages phonetically, using the International Phonetic Alphabet to spell them (sound-based spelling rather than meaning-based spelling). I know several linguists doing research on minority languages (fewer than 10,000 speakers in s…

I would love such a model. My wife's family is Iu Mien which is a sub group of the Dao/Yao Chinese ethnic minority. Mien is its own language but most speakers are essentially illiterate. I'm good with language but there simply isn't a course or any books for learning the language. Not much in writing to begin with given the high illiteracy rate. I would love to build a translation system - project Hail Mary style :)

Re: Transcribe.cpp

#114

Post by sipjca »

rolisz wrote: post #102
sipjca wrote: post #48

Earlier quoted context omitted.

You can fairly easily modify [Handy]( https://handy.computer ) to do this if you want I’m planning on having it as a first class feature of the app too just too many other issues to work on first

Can you give some pointers around this? I'd gladly help with a PR for this, but if you have anything docs/ideas around this it would be helpful.

I’m on a train right now but off the top of my head the audio pipeline may have to be modified slightly to emit partial text segments as they come in from the transcription engine. And then calling the appropriate paste method the user has in their settings.

It may be easier than expected in some way since we already emit events for the live overlay, so it could be as small as a function call, but I don’t know the code path well enough from memory and what complexities it has. Probably with the Tauri context and a bit of other mess we have as this bit of code has gone through a lot of pain

Re: Transcribe.cpp

#115

Post by boomskats »

sipjca wrote: post #110
ctas wrote: post #98

Earlier quoted context omitted.

Really appreciate your work. Is there any way people can help? From your last sentence, it sounds like another PR isn't it and the opposite might be needed. But would love to contribute with testing if helpful. I'm regularly jumping between XFCE, KDE, GNOME, Niri, etc..

Testers by far as the most needed thing, I do maintain a list of per platform people who help to test so if you drop a GitHub username (or email me) I will add you to the list and ping for help Basically the biggest blocker is me being the sole maintainer and reviewer at the moment and it just ends up taking a lot of time for the scale of the project. Which is why it moves slow and features typically are much slower…

I have a personal fork of hyprvoice[0] which I use almost everywhere now (w/ the big cohere-transcribe running on a local vLLM instance). It does a similar thing, but that's not why I'm mentioning it; I think it's worth looking at because it's a clean reference for the few elegant ways you can implement text injection in modern Linux (wayland).

It supports ydotool[1], wtype[2] and "clipboard fallback with clipboard restore". The first two you can probably think of as AHK equivalents - they wire in at the input layer and inject keystrokes when injecting text. wtype is wayland-only and a bit less invasive, ydotool supports non-wayland also apparently, but I haven't tried it. Neither approach provides 'instant text' - you have to watch the text get typed out, and you don't touch your keyboard while it's happening; the clipboard implementation is fallback for a reason as it's the least reliable. The first two work 'well enough' though, and are fairly tunable.

The other thing hyprvoice does in probably the most linux-friendly and universal way is the 'hotkey handling'. The server creates a socket in /tmp that the cli can then ping when the user triggers the start/stop/cancel, and they do this by binding whatever their DE's keyboard shortcut mapping mechanism is to trigger `hyprvoice toggle` as a background shell command. This works extremely well and is much cheaper than you'd intuitively think coming from Windows. This way you don't have to interface with DE-specific global keyboard listeners etc, but leave that to the WM (that's not to say that your installer couldn't prompt the user to configure the keyboard shortcut for them with their detected WM, you just wouldn't do it in the software itself).

I haven't actually looked at your project in too much depth yet as I have a solution for this already, so apologies if none of the above is news to you. Hope it helps though - happy to poke around and contribute something if the gap's still there.

[0]: https://github.com/leonardotrapani/hyprvoice [1]: https://github.com/ReimuNotMoe/ydotool [2]: https://github.com/atx/wtype

Re: Transcribe.cpp

#116

Post by kmfrk »

Well this almost seems to be to good to be true. :)

I assume this is going to make maintaining SubtitleEdit a lot easier from now on, too: https://github.com/SubtitleEdit/subtitleedit/.

Anyone know a good Windows app that's just a window that transcribes - and translates - whatever goes through your output device, and not the microphone like most apps do?

Re: Transcribe.cpp

#117

Post by solarkraft »

Oh, I like this! I’ve been looking into locally hosting a transcription API server and came away feeling pretty close to the problem statement. The things most frequently lacking were streaming support (which I’m so glad this has!) and the support for special words to boost during recognition (which I guess there’s some hope they might add???).

Re: Transcribe.cpp

#119

Post by embedding-shape »

solarkraft wrote: post #117

Oh, I like this! I’ve been looking into locally hosting a transcription API server and came away feeling pretty close to the problem statement. The things most frequently lacking were streaming support (which I’m so glad this has!) and the support for special words to boost during recognition (which I guess there’s some hope they might add???).

> I’ve been looking into locally hosting a transcription API server

I've been hosting my own since whisper.cpp appeared on the scene, thrown up on a server with a 3090ti. Even if there is better/faster stuff out today, it just keeps on working without any issues, the weights are tiny and it's faster than I could need. This is basically what you need to get this working today:

    MODEL="/home/user/projects/ggml-org/whisper.cpp/models/ggml-large-v3-turbo.bin"
    WHISPER_SERVER_BIN="/home/user/projects/ggml-org/whisper.cpp/build/bin/whisper-server"
    "$WHISPER_SERVER_BIN" --model "$MODEL" --language en --host 127.0.0.1 --port 7812
Very simple stuff, throw it on some local homelab server and now you have a local transcription API :) Might need to play around with some of the inference parameters, but once you've locked them in, seems to work really well.

Re: Transcribe.cpp

#120

Post by simsla »

rmunn wrote: post #101

Looks very cool. One thing I have been looking for, which this doesn't seem to cover (at least I didn't see any mention of IPA in the model documentation), is a way to transcribe unknown languages phonetically, using the International Phonetic Alphabet to spell them (sound-based spelling rather than meaning-based spelling). I know several linguists doing research on minority languages (fewer than 10,000 speakers in s…

Automatic Phoneme Recognition (APR). There are some models that do this, but they're only so-so.
Post reply on HN