Can the script be compiled into actual DOM operations so that we don't need LLM for every run？
Show HN: Stagehand – an open source browser automation framework powered by AI
81–89 of 89 posts
Re: Show HN: Stagehand – an open source browser automation framework powered by AI
#82Earlier quoted context omitted.
I was working on a side project over the holidays with the (I think) same idea as mpalmer imagined there too (though my project wouldn't be interested to him either, because my goal wasn't automating tests) Basically, the goal would be to do it like with screenshot regression tests: basically you get 2 different execution phases: - generate - verify And when verify fails in CI, you can automatically run a generate an…
This is super interesting, is it open source? Would love to talk to you more about how this worked
Its not at a stage I'd be comfortable to put it on GitHub yet, maybe in a few months.
And I think you misunderstood my comment, I didn't describe my project, but extrapolated from the parents desire and my motivations for my project.
Mine is actually pretty close to stagehand, at least I could very well use it. It's basically a web UI to configure browser tasks like open webpage x, iterate over "item type", with LLM integration to determine what the CSS selector for that would be. On next execution it would attempt to use the previously determined CSS selector instead of the LLM integration. On failures, it'd raise a notification with an admin tasks to verify new selectors/fix the script
But it's a lot of code to put together as a generic UI - as I want these tasks to be repeatable without restarting from the beginning etc
Still very much in the PoC stage without any tests, barely working persistence etc
Re: Show HN: Stagehand – an open source browser automation framework powered by AI
#83Earlier quoted context omitted.
>Personally I'd love to use this as an intermediate workflow for producing deterministic playwright code, but it looks like this is intended for running directly. Treating UI test code as some kind of static source of truth is the biggest nightmare in all of UI front end development. Web UIs naturally have a ton of "jank" that accumulates over time, which leads to a ton of false negatives; slow API calls, random usag…
> Treating UI test code as some kind of static source of truth is the biggest nightmare in all of UI front end development. Web UIs naturally have a ton of "jank" that accumulates over time, which leads to a ton of false negatives; slow API calls, random usages of websockets/SEE, invisible elements, non-idempotent endpoints, etc. etc. And having to write "deterministic" test code for those is the single biggest reaso…
What's your secret to "proper care and control both the UI and the tests"? If you meant jankiness @ramesh31 mentioned and me in a sibling comment, then that's exactly what I expect for AI tools to solve and achieve a productivity boost.
Re: Show HN: Stagehand – an open source browser automation framework powered by AI
#84Earlier quoted context omitted.
That's not really a problem for Stagehand. It's a problem for Selenium, Playwright, Puppeteer and others at the browser automation library level.
it's not really a problem for Playwrite, because Playwrite is really intended to be run by the owners of the website, not as a webscraper. It may become a real problem for the usefulness of this style of LLM driven browsing.
Playwright does have some docs on scraping, which makes it seem like they do want to support that as a use case https://playwright.dev/docs/docker#crawling-and-scraping. I'm unfamiliar with that though and wouldn't be able to speak to addressing issues with scraping
Re: Show HN: Stagehand – an open source browser automation framework powered by AI
#85Re: Show HN: Stagehand – an open source browser automation framework powered by AI
#86Earlier quoted context omitted.
Hi! Kosta from Octomind here. We built basically this: Let an LLM agent take a look at your web page and generate the playwright code to test it. Running the test is just running the deterministic playwright code. Of course, the actual hard work is _maintaining_ end-to-end tests so our agent can do that for you as well. Feel free to check us out, we have a no-hassle free tier.
Octomind is sick, web agents are such an interesting space; would love to talk to you more about challenges you might've faced in building it
Sorry didnt see this earlier. If you're interested reach out to me (Kosta Welke) on linkedin. Or write me an email, you can find me on Octominds About page.
Re: Show HN: Stagehand – an open source browser automation framework powered by AI
#87Looks interesting. I know about the similar project - https://zerostep.com. Is it basically the same?
Re: Show HN: Stagehand – an open source browser automation framework powered by AI
#88cool extension to playwright! how effective are the ai methods in handling dynamic ui changes?
Re: Show HN: Stagehand – an open source browser automation framework powered by AI
#89Earlier quoted context omitted.
The MCP server is useful, I built a demo of converting bug reports into Playwright tests using it - https://qckfx.com/demo
Whoa -- this is so cool! Is this open source? Would love to check it out
No, but happy to chat about it. My email is chris.wood@qckfx.com