Is there an open source browser RPA that allows mixing of scripted and AI commands? So I could specify exactly what XPath to click on or copy text from mixed with commands like "click the blue button".
Operator research preview
131–140 of 448 posts
Re: Operator research preview
#132Neat, someone should develop an easy-to-deploy script that spawns a headless version of this agent that scrolls through and repeatedly clicks every single ad on X and Facebook using a session cookie.
I love the idea of mucking up ad data, but not a fan of giving free money to X and Meta.
Re: Operator research preview
#133[flagged]
Re: Operator research preview
#134Curious how long this paradigm (computers using human interfaces) will last for P95 tasks. If the machines are smart enough, shouldn’t they be able to build better interfaces to existing software? With that aside, it seems like there are two things at play in this demo: 1. Pixel-tuned GPT-4o 2. “Agent” in prod (supervisor loop + operator loop) Will be interesting to see if they open those up as separate tools in the…
>If the machines are smart enough, shouldn’t they be able to build better interfaces to existing software?
There is no "intelligence" in any of this. Just a whole lot of automation.
Re: Operator research preview
#135> We’re collaborating with companies like DoorDash, Instacart, OpenTable, Priceline, StubHub, Thumbtack, Uber, and others to ensure Operator addresses real-world needs while respecting established norms. Are these tasks really complex enough for people that they are itching to relegate the remaining scrap of required labor to a machine? I always feel like I'm missing something when companies hold up restaurant reserv…
Agree. Most of my imagined use cases involve scraping a nerfed website (e.g. zillow) for data that I can put in a spreadsheet easier use.
Re: Operator research preview
#136Neat, someone should develop an easy-to-deploy script that spawns a headless version of this agent that scrolls through and repeatedly clicks every single ad on X and Facebook using a session cookie.
I love the idea of mucking up ad data, but not a fan of giving free money to X and Meta.
If enough people did it, their entire business model would likely collapse.
Re: Operator research preview
#137Does it read the terms of service or robots.txt before doing stuff?
Re: Operator research preview
#138Earlier quoted context omitted.
I agree with this one. But you have to start somewhere. I think in the next several things, websites will be built for agents and not people. So it'll only get better and smarter.
Will they? What incentive is there if people haven't started using agents yet? We already have a way to build websites for machines: it's called APIs. And frankly, I think that's a better answer for "hooking LLM into website" -- the things which make APIs hard for humans (discomfort, inconvenience, low discoverability, technical complexity) aren't really problems for LLMs.
APIs require devs on both sides.
This kind of thing could just require the devs on one side to maybe clean up a bit of markup (which I even doubt), and the entire universe of potential consumers on the other side.
Re: Operator research preview
#139I wonder, did Google or Microsoft (via Github Copilot) release anything like this yet? I'd not be surprised if all of them are currently working on something in this direction. "Agents", or something like that.
Google has demoed Project Mariner (https://deepmind.google/technologies/project-mariner/), but it is not open to the public yet.
Re: Operator research preview
#140Earlier quoted context omitted.
I assume here it means complying with requests that could harm other people. It's pretty common for businesses to tell their employees not to assist customers doing bad things, so not surprised to see AIs trained to not to assist customers doing bad things. Examples: - "operator, please sign up for 100 fake Reddit accounts and have them regularly make posts praising product X." - "operator, please order the component…
It's pretty troubling and illiberal to use the same word for a software tool being constrained by its manufacturer's moral framework and for a human user being constrained to that manufacturer's moral framework. While you can see how the word is formally valid and analogous in both cases, the connotation is that the user is being judged by the moral standards of a commercial vendor, which is about as Cyberpunk Dystop…
This is putting it in better words than I came up with myself.