Ask HN: What are you working on? (March 2025)

This is at the very early stages where I have a design sketch and some experiments that validate the design. Below is the README:

Rio is an experimental C++ async framework. The goal is to provide a lightweight framework for writing C++ server applications that is easy to use and provides low consistent latencies.

Today, async frameworks that focus on efficiency typically use one of two architectures:

1. Shared nothing architectures, also called thread-per-core. This is used by frameworks such as Seastar and Boost.Asio. In a shared-nothing architecture, each worker thread runs its own event loop and is intended to run on its own dedicated core. The application is architected to shard its workload over multiple workers with only infrequent communication between them. When a task performs a CPU bound task, it needs to be explicitly run in a thread pool as otherwise it would block other tasks from running in the current worker (often referred to as a "reactor stall").

2. Work stealing architectures. This architecture is used by frameworks such as Tokio. In this case there are also multiple worker threads, each running their own event loop. When a specific worker gets overloaded or runs a blocking task, other threads can execute ready tasks. This goes some way to prevent reactor stalls. However, even though other threads can steal ready tasks, they do not poll the event loop for new readiness events. This means that a task that does not yield back to the runtime will increase latencies for other requests assigned to that worker.

The thesis for Rio is that in real-world server applications, it gets increasingly hard to ensure you yield back frequently to the event loop. In particular, there are many CPU bound tasks that server applications commonly perform, such parsing protocols, or performing encryption and compression. If tasks take less than ~10 microseconds it is often not worth it to offload these to a thread pool as the system call overhead of synchronization will take more time than this. Additionally, putting in various thread offloads makes it harder to develop, especially in larger teams with individuals of different experience levels. The result is that there will either be too many work pushed on thread pools or too little. The net effect will be that latencies will be less consistent.

Rio is an experiment for a work stealing architecture where completion events can also be stolen. The Rio runtime uses multiple worker threads to handle asynchronous tasks. Each worker threads runs its own io_uring, which is also registered to an eventfd. A central "stealer" thread listens to the eventfds for all workers. When an readiness event becomes available, the stealer will check if the corresponding thread is currently executing a task. If so, it will signal an idle worker with a request to process the completion event and run any tasks that results in it. The stealing logic is aware of the system topology and will try to wake up a thread that shares a higher level cache with the task.

It's essentially a small search engine for videos that runs locally on your laptop. Previously, the system just extracted information about whole video files and maintained a searchable list of those files.

I'm putting the finishing touches on a major architectural change to solve a request from someone who creates highlight reels for professional soccer matches. They needed to tag and search for specific moments within videos - like all goals scored by a particular player across dozens of camera angles of dozens of matches.

This required re-engineering the entire data model to support metadata entries that point to specific segments of a file entity rather than just the file itself.

Instead of treating each file as an atomic unit, I now separate the concept of "content" from the actual files that contain it. This distinction allows me to:

1. Create "virtual clips" - time segments within a video that have their own metadata, tags, and titles - without generating new files

2. Connect multiple files that represent the same underlying content (like a match highlight with different ad insertions for YouTube vs. Twitch)

3. Associate various resolution encodings with the same logical content

For example, a content creator might have multiple versions of the same video with different ad placements for different channels. In the old system, these were just separate, unrelated files. Now, they're explicitly modeled as variants of the same logical content, making organization and search much more powerful.

I also completely reworked the job system, moving from code running in the Electron main process to spawning Temporal in a child process. I know it sounds like overkill for a desktop app, but it's been surprisingly perfect for a desktop app that's processing hundreds of terabytes of video.

When you're running complex ML pipelines on random people's home computers, things go wrong - custom FFmpeg versions that don't support certain codecs, permission issues with mounted volumes, their local model server crashes. With Temporal, when a job fails, I just show users a link to the Temporal Web UI where they can see exactly what failed and why. They can fix their local config and retry just that specific activity instead of starting over. It's cut my support burden dramatically.

My developer experience is so much better too. The ML pipeline for face recognition has multiple stages (small model looks for faces, bigger model does pose detection, embedding generation with even larger model) that takes minutes to run. With Temporal, I can iterate on just the activity that's failing without rerunning the entire pipeline. Makes developing these data-intensive workflows so much more manageable.