Launch HN: Augento (YC W25) – Fine-tune your agents with reinforcement learning

Well... we took the rawness to heart, that's clear!

I want to make sure I am understanding this.

If I have an application that uses OpenAI models then this service can act as a proxy between the my application and the actual OpenAI service. It logs all of the requests that get sent to the OpenAI api. At some later time, I can go through and choose a subset of the API calls and mark them (I'm guessing as good or bad) and these get converted into a training set. I then have to create a value function as its own API that I run on my own servers somewhere (like fly.io). Then I start a training run, which I assume will use some open source AI model to regenerate responses to the training set derived from my initial OpenAI api calls. It then takes the generated responses from that open source model, sends them to my value function api which scores them, and then uses that score to apply some RL magic to the base open source model. At the end of this process I have an open source model that has been RL trained based on the captured api calls as well as the scoring from the value function.

I suppose the argument here is, a RL trained open source model will perform your task better than the base OpenAI model. So your target market is, people already using OpenAI api, they have the desire and funds to experiment with RL, they have the capability of defining a value function, they are able to sift through their api calls to identify the ones that aren't performing well and isolate them, and they are willing to swap out their OpenAI model with an open source model that is RL trained if it can be shown it is more accurate.

I would guess this market exists and the need is real. Defining a value function is much easier than building the infrastructure to RL a variety of open source models. So someone who wants to do this may appreciate paying for someone else who has already set up the infrastructure. And they don't want to host their own model (their already paying for OpenAI model hosting) so maybe they have no problem paying you for inference as well.

Whether or not this succeeds as a business really depends on how effective RL is for the clients you find. There are two paths here, RL is wildly successful and therefore so are you. Or RL fine-tuning is unable to keep up with foundation model advancements and clients will learn it is better to wait it out on the big fellas rather than go through the time-consuming and costly process.

Congrats on the launch!

Noob question - from my understanding, SoTA proprietary models already provide APIs for fine tuning, I'd say it's only a matter of time before they provide RL based APIs, no?

We could open-source (parts of) our platform. What specifically would you like to see open-sourced? We thought about developing this into a piece of software you can run in your own cloud (for compliance and security) but at the moment this makes the GPU economics really difficult and would probably be only interesting/relevant to big enterprises. Anyway, we're definitely curious to hear if anyone has interesting applications for an open-source version of Augento!