Is this solution similar to the Direct Preference Optimization (DPO) [1] provided by another 'fine-tuning as a service' - OpenPipe?
Launch HN: Augento (YC W25) – Fine-tune your agents with reinforcement learning
41–50 of 63 posts
Re: Launch HN: Augento (YC W25) – Fine-tune your agents with reinforcement learning
#42Lol, can't use in non-subscription mode. It requires subscription to import provider.
Re: Launch HN: Augento (YC W25) – Fine-tune your agents with reinforcement learning
#43Lol, can't use in non-subscription mode. It requires subscription to import provider.
Hi! You won't get billed for importing a provider. You just need a user account because your providers need to be associated to your Augento user. You can then start to use the data ingestion onto the platform - free of charge, of course ;) Actual billing then applies only to training and inference.
Re: Launch HN: Augento (YC W25) – Fine-tune your agents with reinforcement learning
#44Earlier quoted context omitted.
That’s fair! It has been mentioned before so we‘ll likely build that into the platform. Would you like us to upload your model to your huggingface account, download the weights or choose an inference provider we then upload it to?
I (not GP) would like to be able to choose between the options. Inference provider isn't super necessary though (can do that through huggingface).
Thanks for stating your preference! This is something we can incorporate into the platform.
Re: Launch HN: Augento (YC W25) – Fine-tune your agents with reinforcement learning
#45I want to make sure I am understanding this. If I have an application that uses OpenAI models then this service can act as a proxy between the my application and the actual OpenAI service. It logs all of the requests that get sent to the OpenAI api. At some later time, I can go through and choose a subset of the API calls and mark them (I'm guessing as good or bad) and these get converted into a training set. I then…
Does it mean after I successfully trained the Open Source Model, I don't need OpenAI anymore?
Re: Launch HN: Augento (YC W25) – Fine-tune your agents with reinforcement learning
#46I want to make sure I am understanding this. If I have an application that uses OpenAI models then this service can act as a proxy between the my application and the actual OpenAI service. It logs all of the requests that get sent to the OpenAI api. At some later time, I can go through and choose a subset of the API calls and mark them (I'm guessing as good or bad) and these get converted into a training set. I then…
Does it mean after I successfully trained the Open Source Model, I don't need OpenAI anymore?
Yes, indeed
Re: Launch HN: Augento (YC W25) – Fine-tune your agents with reinforcement learning
#47Also I think if I'd use your product, I'd like to be able to host the model elsewhere in case I don't like the platform anymore :)
That’s fair! It has been mentioned before so we‘ll likely build that into the platform. Would you like us to upload your model to your huggingface account, download the weights or choose an inference provider we then upload it to?
Seems like download the weights would be the most flexible option. HF and inference providers would be nice to have.
Re: Launch HN: Augento (YC W25) – Fine-tune your agents with reinforcement learning
#48I love the idea of the product! I would trust your solution to be the best for very simple use cases but not for multistep or ReAct agents. Any thoughts / insights on that? I think the demo could be more exciting, the voice of the person talking sounds like he's bored haha
Ha - here's the advice I give to YC startups about making demo videos for HN: "What works well for HN is raw and direct, with zero production values. Skip any introductions and jump straight into showing your product doing what it does best. Voiceover is good, but no marketing slickness—no fancy logos or background music!" I guess there's zero production values and zero production values...
Totally agree. Raw is great, but energy matters too. If the person sounds bored, it's hard to get excited about the product—even if it's amazing. Passion is contagious.
Re: Launch HN: Augento (YC W25) – Fine-tune your agents with reinforcement learning
#49This is just dev ops wrapped around an open source fine tuning repo.
It's convenience. And people pay for convenience all the time.
People pay for convenience, that's true - and part of the equation here. Agreed! The approach is to make data capturing as convenient as possible, where you just paste in api key + base url into your existing code, and you gather all your runs. And then, Reinforcement Learning is hard to figure out - so one of the goals is to commoditize Reinforcement Learning, what you're alluding to. In its iteration, the platform is released with verifiable mode where Augento takes all the headache of GPU infrastructure, GRPO implementation, training configurations and dataset curation away - you just select your gathered runs, and start the training. But we'll go past that, and expand Augento into a platform for alignment and self-learning.
Tl;DR Yes, indeed! We designed Augento with convenience in mind.
Re: Launch HN: Augento (YC W25) – Fine-tune your agents with reinforcement learning
#50This looks great. I have a few questions. 1. I'm assuming by the pricing it's "serverless" inference, what's the cold-start time like? 2. Any idea on inference costs? Also just to reiterate what others say but the option of exporting weights would definitely make it more appealing (although it sounds like that's in the roadmap).
Thanks!
> I'm assuming by the pricing it's "serverless" inference, what's the cold-start time like?
Yeah, you could probably call it serverless inference. However, due to the fact that all fine-tuned models are trained on the same base model(s), we have some interesting optimizations we can apply over standard "serverless" model deployment. The biggest is that we can keep the base model loaded in VRAM and only swap the trained weight deltas per request. This gives us sub-second cold-start times for inference in the average case.
> Any idea on inference costs?
Right now, we’re pricing inference at $0.5/M input tokens, $2.5/M output tokens. That’s in a similar price range but a bit lower than gpt-4o/Claude 3.5, which we consider the main models we’re "competing" with. As it’s our goal to democratize access to models/agents in the long run, we hope that we can drop the prices for inference further, which should be enabled by some other optimizations we’re currently planning.