Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: I wrote an open-source browser alternative for Computer Use for any LLM

github.com

Post reply on HN
Print view View original

61–70 of 75 posts

Re: Show HN: I wrote an open-source browser alternative for Computer Use for any LLM

#61

Post by Tepix »

atrus wrote: post #58
gitgud wrote: post #57

It's impressive, but to me it seems like the saddest development experience... agent = Agent( task='Go to hackernews on show hn and give me top 10 post titels, their points and hours. Calculate for each the ratio of points per hour.', llm=ChatOpenAI(model='gpt-4o'), ) await agent.run() Passing prompts to a LLM agent... waiting for the black box to run and do something...

I mean, is that really much different than an API? I pass a query, and get data back, and rarely do I get to inspect the mechanisms behind what's returning that data.

Well, i guess that with an LLM you kind of hope it will understand you whereas with a regular program you don't have to hope for that, just that it will work without throwing errors (usually with a magnitudes lower error rate).

Re: Show HN: I wrote an open-source browser alternative for Computer Use for any LLM

#63

Post by gregpr07 »

gitgud wrote: post #57

It's impressive, but to me it seems like the saddest development experience... agent = Agent( task='Go to hackernews on show hn and give me top 10 post titels, their points and hours. Calculate for each the ratio of points per hour.', llm=ChatOpenAI(model='gpt-4o'), ) await agent.run() Passing prompts to a LLM agent... waiting for the black box to run and do something...

For that we wanted to give you more control as well with a for loop and you take actions step by step. I think all of these crewai like agent swarms are also very much black boxes.

How would you imagine the perfect scenario? What would make LLM outputs less of a black box?

Re: Show HN: I wrote an open-source browser alternative for Computer Use for any LLM

#64

Post by stuartjohnson12 »

maggreenWAI wrote: post #47

Let's say in 1 year, more agents than humans interact with the web. Do you think: 1. Websites release more API functions for agents to interact with them or 2. We will transform with tools like this the UI into functions callable by agents and maybe even cache all inferred functions for websites in a third party service?

Came across some folks a little while ago who just raised a few million $ for the latter (in stealth).

Re: Show HN: I wrote an open-source browser alternative for Computer Use for any LLM

#65

Post by gregpr07 »

stuartjohnson12 wrote: post #64
maggreenWAI wrote: post #47

Let's say in 1 year, more agents than humans interact with the web. Do you think: 1. Websites release more API functions for agents to interact with them or 2. We will transform with tools like this the UI into functions callable by agents and maybe even cache all inferred functions for websites in a third party service?

Came across some folks a little while ago who just raised a few million $ for the latter (in stealth).

Who (or still in stealth haha)?

Re: Show HN: I wrote an open-source browser alternative for Computer Use for any LLM

#66

Post by gregpr07 »

DeathArrow wrote: post #62

>This enables you to design custom web automation and scraping functions without manual inspection through DevTools Can it use a headless browser?

Technically you can yeah, I am not sure if the performance is the same - would have to test it!

Re: Show HN: I wrote an open-source browser alternative for Computer Use for any LLM

#67

Post by daft_pink »

craftkiller wrote: post #15
daft_pink wrote: post #13

I was really excited about the original claude computer use until I watched the youtube videos and saw it was only running in a docker container. I wish I could run something like this on a real machine.

What makes a docker container not a "real machine"? Docker programs are running natively, just like any other program, without emulation/virtualization. Its not like a virtual machine (unless you're on one of the lesser operating systems like Windows or OSX), its just configuring some settings in the Linux kernel to isolate the process from other processes. Its basically just an enhanced chroot.

Because I would like to automate programs I use in Windows with files that I actually use, instead of some random linux docker container. I should have used my actual machine instead of a real machine.

Re: Show HN: I wrote an open-source browser alternative for Computer Use for any LLM

#68

Post by stuckkeys »

arjvik wrote: post #51
gregpr07 wrote: post #10

Earlier quoted context omitted.

Not sure, I think there is a lot of research being done here. Actually, browser use works quite well with vision turned off, it just sometimes gets stuck at some trivial vision tasks. The interesting thing is that screenshot approach is often cheaper than cleaned up html, because some websites have HUGE action spaces. We looked at some papers (like ferret ui) but i think we can do much better on html tasks. Also, the…

Would be really cool if you could tie this into Claude's computer use APIs!

My wallet just ran way. Come back you!

Re: Show HN: I wrote an open-source browser alternative for Computer Use for any LLM

#69

Post by stuckkeys »

gregpr07 wrote: post #50
EGreg wrote: post #48

Earlier quoted context omitted.

Aren't screenshots far larger than this?

Nope: 1280x1024 low resolution with gpt-4o are 85 tokens so approx $0.0002 (so 100x cheaper). For high resolution its apporx $0.002 https://openai.com/api/pricing/

Yeah. I noticed a very low cost when I run it via vm, predefined resolution. Good tip.

Re: Show HN: I wrote an open-source browser alternative for Computer Use for any LLM

#70

Post by craftkiller »

daft_pink wrote: post #67
craftkiller wrote: post #15

Earlier quoted context omitted.

What makes a docker container not a "real machine"? Docker programs are running natively, just like any other program, without emulation/virtualization. Its not like a virtual machine (unless you're on one of the lesser operating systems like Windows or OSX), its just configuring some settings in the Linux kernel to isolate the process from other processes. Its basically just an enhanced chroot.

Because I would like to automate programs I use in Windows with files that I actually use, instead of some random linux docker container. I should have used my actual machine instead of a real machine.

> Windows

Ah, well there's your problem. Your problem isn't docker, nor is it claude, its that you're running Windows.

Post reply on HN