Earlier quoted context omitted.
Yes, this is the pass@k metric from code generation research. Found the relevant paper Evaluating Large Language Models Trained on Code (Chen et al., 2021) which introduced the metric.
Interesting, and how does Twill uses it in that feature?
Launch HN: Twill.ai (YC S25) – Delegate to cloud agents, get back PRs
31–40 of 113 posts
Re: Launch HN: Twill.ai (YC S25) – Delegate to cloud agents, get back PRs
#3224/7 running coding agents are pretty clearly the direction the industry is going now. I think we'll need either on-premises or cloud solutions, since obviously if you need an agent to run 24/7 then it can't live on your laptop. Obviously cloud is better for making money, and some kind of VPC or local cloud solution is best for enterprise, but perhaps for individual devs, a self-hosted system on a home desktop comput…
Re: Launch HN: Twill.ai (YC S25) – Delegate to cloud agents, get back PRs
#33Something very useful that will be harder for you most likely is code search. Having a proper index over hundreds of code repos so the agent can find where code is called from or work out what the user means when they use an acronym or slightly incorrect name.
It's quite nice to use and I'm sure someone will make a strong commercial offering. Good luck
Re: Launch HN: Twill.ai (YC S25) – Delegate to cloud agents, get back PRs
#34Re: Launch HN: Twill.ai (YC S25) – Delegate to cloud agents, get back PRs
#35Earlier quoted context omitted.
Not sure if you've seen it yourself but Claude code can kick off parallel agents working in their own worktrees natively now. I do it all the time.
Yes, the difference is that Twill launches dedicated infra on each sandbox for each task. This means you can work on multiple tasks requiring a DB migration for instance. Also you can fire and forget tasks (my favorite) and don't have to keep your laptop running at night.
See also Show HN, this exact product is frequently shown as a github link.
The paradigm shift in Ai means what you are making is (1) filling a gap until the primaries implement it, most have it in their pipeline if not already (2) how easy it is to replicate with said Ai using my preferred tech stack
Re: Launch HN: Twill.ai (YC S25) – Delegate to cloud agents, get back PRs
#36I built an internal version of this for my workplace. Something very useful that will be harder for you most likely is code search. Having a proper index over hundreds of code repos so the agent can find where code is called from or work out what the user means when they use an acronym or slightly incorrect name. It's quite nice to use and I'm sure someone will make a strong commercial offering. Good luck
That said, there are workarounds, like cloning all repos and enabling LSP (coding CLIs added that feature) or using a dedicated solution for codebase indexing and add a skill/mcp.
Super fast models spamming grep commands are also fun to watch!
Curious to know how you implemented it in house.
Re: Launch HN: Twill.ai (YC S25) – Delegate to cloud agents, get back PRs
#37How does this compare to Jules from Google?
- Twill is CLI-agnostic, meaning you can use Claude Code, Codex or Gemini. Jules only works with Gemini.
- We focus on the delegation experience: Twill has native integrations with your typical stack like Slack or Linear. The PRs comes back with proofs of work, such as screenshots or videos.
Re: Launch HN: Twill.ai (YC S25) – Delegate to cloud agents, get back PRs
#38Re: Launch HN: Twill.ai (YC S25) – Delegate to cloud agents, get back PRs
#39Earlier quoted context omitted.
Yes, the difference is that Twill launches dedicated infra on each sandbox for each task. This means you can work on multiple tasks requiring a DB migration for instance. Also you can fire and forget tasks (my favorite) and don't have to keep your laptop running at night.
See also Cowork and other upcoming Anthropic features. See also Show HN, this exact product is frequently shown as a github link. The paradigm shift in Ai means what you are making is (1) filling a gap until the primaries implement it, most have it in their pipeline if not already (2) how easy it is to replicate with said Ai using my preferred tech stack
What Anthropic can't offer is to let you use Codex or combine it with Claude Code. That is why we think non ai-labs players have a say in this market.
To your last point, as always there is a buy vs build tradeoff which ultimately comes down to focusing on your core business which we think still remains important in the ai era
Re: Launch HN: Twill.ai (YC S25) – Delegate to cloud agents, get back PRs
#40> Run the same agent n times to increase success rate. Are there benchmarks out there that back this claim?