Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ace: Realtime Computer Autopilot

generalagents.com

Post reply on HN
Print view View original

1–10 of 22 posts

Re: Ace: Realtime Computer Autopilot

#3

Post by xfr »

sherjilozair wrote: post #2

I'm the founder and CEO of General Agents. Happy to answer questions!

First, I am extremely impressed by the demo. It looks truly groundbreaking.

Could you elaborate on the types of tasks and data sources used to train Ace, and how these contribute to its performance on desktop automation?

Ace is said to outperform other models on your suite of computer use tasks. Can you provide more details on these benchmarks and how Ace compares to existing automation tools?

Re: Ace: Realtime Computer Autopilot

#5

Post by chews »

sherjilozair wrote: post #2

I'm the founder and CEO of General Agents. Happy to answer questions!

From your site, "Ace works like we do—performing mouse clicks and keystrokes based on the screen and prompt—trained with Is there a way to train or augment training on applications you've never seen before? We have a bunch of custom Java applications that we use in finance, curious about some additional automation.

Re: Ace: Realtime Computer Autopilot

#6

Post by sherjilozair »

chews wrote: post #5
sherjilozair wrote: post #2

I'm the founder and CEO of General Agents. Happy to answer questions!

From your site, "Ace works like we do—performing mouse clicks and keystrokes based on the screen and prompt—trained with Is there a way to train or augment training on applications you've never seen before? We have a bunch of custom Java applications that we use in finance, curious about some additional automation.

Ace is actually uniquely designed to support that. Our training staff simply record their screens and mouse+keyboard events. We transform that into behavior cloning data to train the model. It's quite easy for us to do custom agents for enterprise or other lesser-known software and workflows. Reach out to us at contact@generalagents.com if you're interested.

Re: Ace: Realtime Computer Autopilot

#7

Post by gkolli »

sherjilozair wrote: post #2

I'm the founder and CEO of General Agents. Happy to answer questions!

Hi! Looks pretty interesting - few questions/thoughts:

1. Could you talk a bit more about your behavioral-training? If ace-control is trained on behavioral recordings, would it choose the most efficient path for the agent to take to complete a task? I'm guessing humans choose naturally take less-optimal steps.

2. What causes the huge speed increase? I'm guessing there were a lot of optimizations made, especially since this behavioral-training seems very different from vision models. I'm guessing the model is smaller, so it's interesting that accuracy is highest. I'd be interested to see a comparison vs. 4o-mini

3. Would be neat for it to handle instructions offline/locally - like "connect me to wifi" ;)

4. Would be cool if agent could work in the background so I can do something else in the meantime. ;)

Post reply on HN