Interacting on the pixel level feels as circuitous as rendering text to hardcopy, manually annotating it, and then digitizing it back through OCR.
Operator research preview
91–100 of 448 posts
Re: Operator research preview
#92Earlier quoted context omitted.
[flagged]
Then why not say that in the first place? It's not very charitable to get upset with someone for correcting an incorrect number you yourself stated. Also, what's up with the faux-stutter?
Why? The official announcement is a $500B commitment. Even if they "only get $100B" (which I doubt) do you think that will somehow be detrimental to OpenAI?
Do you understand how much money a hundred billion dollars is?
Re: Operator research preview
#93Why a US-only release? Have they done that for other research previews?
Wonder what’s changed recently..
Re: Operator research preview
#94They also just announced o3-mini will be on free tier for chatGPT as well.
Have they talked about which tiers of o3-mini they'll use for which plan?
Re: Operator research preview
#95I'll just say from a demo perspective: Bold move using presumably real email addresses and credit cards on a live stream like this. I feel bad for that restaurant since I'm sure some jokers were trying to reserve all the table as soon as it popped up on screen.
Re: Operator research preview
#96I don't know why, but the approach where "agents" accomplish things by using a mouse and keyboard and looking at pixels always seemed off to me. I understand that in theory it's more flexible, but I always imagined some sort of standard, where apps and services can expose a set of pre-approved actions on the user's behalf. And the user can add/revoke privileges from agents at any point. Kind of like OAuth scopes. Ima…
> but I always imagined some sort of standard, where apps and services can expose a set of pre-approved actions on the user's behalf I sincerely hope it's not the future we're heading to (but it might be inevitable, sadly). If it becomes a popular trend, developers will start making "AI-first" apps that you have to use AI to interact with to get the full functionality. See also: mobile first.
Why would developers do that?
The developer's incentive is to control the experience for a mix of the users' ends and the developer's ends. Functionality being what users want and monetization being what developers want. Devs don't expose APIs for the same reason why hackers want them - it commodifies the service.
An AI-first app only makes sense if the developer controls the AI and is developing the app to sell AI subscriptions. An independent AI company has no incentive to support the dev's monetization and every incentive to subvert it in favor of their own.
(EDIT: This is also why AI agents will "use" mice and keyboards. The agent provider needs the app or service to think they're interacting with the actual human user instead of a bot, or else they'll get blocked.)
Re: Operator research preview
#97I don't know why, but the approach where "agents" accomplish things by using a mouse and keyboard and looking at pixels always seemed off to me. I understand that in theory it's more flexible, but I always imagined some sort of standard, where apps and services can expose a set of pre-approved actions on the user's behalf. And the user can add/revoke privileges from agents at any point. Kind of like OAuth scopes. Ima…
> I always imagined some sort of standard, where apps and services can expose a set of pre-approved actions on the user's behalf OS specific, but Apple has the Scripting Support API [0] and Shortcut API for their app. Works great. [0]: https://developer.apple.com/documentation/foundation/scripti...
Yep, and on Windows this is exposed through the COM api.
Re: Operator research preview
#98Earlier quoted context omitted.
Big jumps in benchmarks from Claude's Computer Use though. 87% vs 56% on Webvoyager 58.1% vs 36.2% on WebArena 38.1% vs 22% on OsWorld These are next gen improvements so the fact that Claude didn't make any waves doesn't really mean anything (Of course no guarantee this will either)
OpenAI is merely matching SOTA in browser tasks as compared to existing browser-use agents. It is a big improvement over Claude Computer Use, but it is more of the same in the specific domain of browser tasks when comparing against browser-use agents (which can use the DOM, browser-specific APIs, and so on.) The truth is that while 87% on WebVoyager is impressive, most of the tasks are quite simple. I've played with…
Well that's fair. I wasn't saying that this was necessarily at a level of competence to be useful, simply that it seemed to be a lot better than Claude.
Re: Operator research preview
#99I think one of the things I'm most excited for is that this really opens up, for practical purposes, a lot of websites that made it difficult to do things via API. For example, while I frequently end up booking AirBnB's, I find the process of searching for an AirBnB quite tedious.
I dream of a world where I can specify annoying things to me and build a perfect search for any house, that understands how I think about money, how I think about my family, and what I love and really extends how I interact with the world.
Re: Operator research preview
#100We already have this https://github.com/browser-use/browser-use
The advantage of this repo is that it doesn't require models to output click coordinates.