Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Audioscrape: Building in Rust When Everyone Said I Shouldn't

news.ycombinator.com

Post reply on HN
Print view View original

21–30 of 47 posts

Re: Audioscrape: Building in Rust When Everyone Said I Shouldn't

#21

Post by katangafor »

echelon wrote: post #9

Love your story here! You're not doing it wrong at all. > Has anyone else bootstrapped using Rust? What were your experiences? We scaled several websites to 6M+ MAUs on Actix/SQLx and ran that on the lowest tier VMs. Rust is God Mode for efficient and scalable HTTP services. Moreover, the type safety and robust type system give us incredible confidence that our software is defect free.

That is absolutely wild!!

I've put off learning rust for so long. I really tried to get into it once, and ended up learning clojure instead lol. Rust just had too many concepts at once for me to learn, probably because I've only ever used garbage-collected languages.

But hearing this type of stuff always makes me want to try again

Re: Audioscrape: Building in Rust When Everyone Said I Shouldn't

#23

Post by samatman »

lukaesch wrote: post #18
aleksjess wrote: post #5

4k loc in a single file must be brutal

Thanks! I was inspired by levelsio's meme about having MVPs in a single index.php file. Traditionally, I've organized codebases into folders, but I started questioning its necessity. Folders often just add an extra layer to search through. It's basically a search param. With Neovim and strict naming conventions, I've found managing everything in one file works quite well. Keyboard navigation can make folders feel lik…

Might want to check out Harpoon: https://github.com/ThePrimeagen/harpoon/tree/harpoon2

No comment on your one-file approach one way or the other, but like all of us, you will need to deal with tree-shaped projects, I've found Harpoon to be a good solution for this. Global marks can only get you so far.

Good luck with Audioscape btw.

Re: Audioscrape: Building in Rust When Everyone Said I Shouldn't

#24

Post by lyang2821 »

I bootstrapped Lapdev (https://lap.dev/) all in Rust too. Axum for the backend and Leptos for the frontend. Sharing structs between backend and frontend is quite nice.

Besides the memory safety, type system etc, Rust gives you the confidence that you know it can scale in pretty much all scenarios in terms of performance.

Re: Audioscrape: Building in Rust When Everyone Said I Shouldn't

#25

Post by lukaesch »

aeturnum wrote: post #22

Neat! Just fyi I tried to add Criminal[1] and it seems like it was...partially successful[2]? The "Enter RSS URL" page loaded for a while and then stopped. [1] https://thisiscriminal.com/episodes [2] https://www.audioscrape.com/podcast/criminal

Thanks for the hint!

We hit the front page on Hacker News and received many submissions, which led to network errors when fetching some podcasts. I've just added retry logic and scheduling for new podcast fetches and pushed these changes to production.

The Criminal[1] podcast is now added. Due to the current demand, our transcription queue is a bit backed up, so transcription for this podcast will appear in a few days. Please be invited to have a look the next days again.

I'm actively working on securing more GPUs to help scale this process efficiently so in the future you don't have to wait that long.

Would it be interesting if I add e-mail "transcription ready" notifications for those who submit a new podcast?

[1] https://www.audioscrape.com/podcast/criminal

Re: Audioscrape: Building in Rust When Everyone Said I Shouldn't

#27

Post by aeturnum »

lukaesch wrote: post #25
aeturnum wrote: post #22

Neat! Just fyi I tried to add Criminal[1] and it seems like it was...partially successful[2]? The "Enter RSS URL" page loaded for a while and then stopped. [1] https://thisiscriminal.com/episodes [2] https://www.audioscrape.com/podcast/criminal

Thanks for the hint! We hit the front page on Hacker News and received many submissions, which led to network errors when fetching some podcasts. I've just added retry logic and scheduling for new podcast fetches and pushed these changes to production. The Criminal[1] podcast is now added. Due to the current demand, our transcription queue is a bit backed up, so transcription for this podcast will appear in a few day…

Oh yah I totally understand the current circumstances, just trying to be helpful in debugging.

I think a notification feature makes a lot of sense, but it depends on how long the wait tends to be. In terms of setting expectations it might be better to display the current backlog and an estimate about when the transcription might be done (though of course both would be even better than either).

Re: Audioscrape: Building in Rust When Everyone Said I Shouldn't

#28

Post by jamil7 »

Awesome work. Yes, I’ve been interested in Rust for a while and have been hacking on a side project. It’s an API service in Rust, but for the CRUD management part I used Elixir/Phoenix and Phoenix gen auth to have something up quickly that’s not dependent on some third-party service.

Re: Audioscrape: Building in Rust When Everyone Said I Shouldn't

#29

Post by lukaesch »

jamil7 wrote: post #28

Awesome work. Yes, I’ve been interested in Rust for a while and have been hacking on a side project. It’s an API service in Rust, but for the CRUD management part I used Elixir/Phoenix and Phoenix gen auth to have something up quickly that’s not dependent on some third-party service.

Thanks. Interesting. What are some best practices for integrating Rust with Elixir/Phoenix in a project, especially when managing CRUD operations and maintaining system efficiency you learned along your side project?

Re: Audioscrape: Building in Rust When Everyone Said I Shouldn't

#30

Post by lukaesch »

miki123211 wrote: post #26
lukaesch wrote: post #10

Earlier quoted context omitted.

whisper + pyannote + custom LLM pipeline to reduce errors, extract and attach speaker labels

How do you interface with the Python ML stack from Rust?

Whisper and Pyannote via Replicate to spin up a container for transcription and diarization. It returns word-level timestamps and speaker turns.

The LLM pipeline is built into the Rust code and makes calls to the OpenAI API.

Post reply on HN