Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Autotab – An AI-powered Chrome extension to create Selenium scripts

autotab.com

Post reply on HN
Print view View original

71–80 of 131 posts

Re: Show HN: Autotab – An AI-powered Chrome extension to create Selenium scripts

#71

Post by jonasnelle »

monkpit wrote: post #51
alexirobbins wrote: post #26

Earlier quoted context omitted.

Selectors have been our primary focus so far – they're notoriously finicky! Our roadmap includes more extensive use of AI, both as embedded intelligence, and in the code generation process. For example, one thing we've heard from heavy users of browser automation is that maintenance becomes the largest cost. Self-healing automations will be able to either fix themselves, our give you an alert with a suggested fix to…

So I assume the video is the ground-truth, then the AI has access to the DOM and the video, and generates a selector based on the video during the test run (each time) in order to do avoid flakiness due to DOM/class/attribute changes?

Right now the generated script is the ground truth but we’ve been working on augmenting this with images & videos to fall back on. We think defaulting to code is good because it is faster, cheaper and more easy to reason about in the 95%+ of times it works. Plain old Selenium will get you pretty far, especially if creating scripts is much easier.

Re: Show HN: Autotab – An AI-powered Chrome extension to create Selenium scripts

#72

Post by tmerse »

bdcravens wrote: post #22
abdusco wrote: post #10

Earlier quoted context omitted.

Playwright is another, really good alternative

It is, but the selector issues are the same. (though it does make it much easier to execute scripts in the context of the page when standard selectors don't work)

It is, but the selector issues are the same

I disagree. Playwright's locators are pretty powerful compared to standard css/xpath selectors.

For example it includes layout based selectors.

https://playwright.dev/docs/other-locators#css-matching-elem...

Re: Show HN: Autotab – An AI-powered Chrome extension to create Selenium scripts

#73

Post by omarfarooq »

What would make this maximally useful is if an LLM with a ruleset would be available to parse and act on dynamic situations on pages.

For example, add to cart, if the LLM detect it's out of stock (a rule), then do some other action.

What's the possibility of LLM-reinforced, rules based branching logic like this being possible with your software in the future?

Re: Show HN: Autotab – An AI-powered Chrome extension to create Selenium scripts

#74

Post by jasonjmcghee »

jonasnelle wrote: post #31
pid-1 wrote: post #9

Related: https://playwright.dev/ has a built-in code generator. In *my opinion*, it's also more pleasant to work with than Selenium.

Interesting, what features are most impactful in making it more pleasant to work with? We picked Selenium because it seems much more widely used at least in the Python world (selenium has about 10x more downloads on PyPI than playwright). My guess is that's at least partly because Playwright is more focused on the JS/TS ecosystem and testing.

Wildly better developer experience. Playwright is also much less flakey and easier to work with than cypress, another alternative to selenium.

Selenium is quite outdated, though still widely used. It's much older. I would look at downloads over time / rate of change, not total downloads.

My hot take- don't hook yourselves to outdated tech.

Don't take my word for it- go try out selenium, Cypress, and playwright and draw your own conclusions.

Re: Show HN: Autotab – An AI-powered Chrome extension to create Selenium scripts

#75

Post by lobochrome »

jonasnelle wrote: post #60
derac wrote: post #50

I'd love if it could output Playwright code as well. I haven't tried it but if this works well it's a great concept!

Good to know! The comments have definitely made us update towards looking into Playwright again quite seriously

Yeah - I moved my entire automation library over at the beginning of the year. The docker integrations and ease of switching out browsers without having to manage webdrivers individually sealed the deal. I'm also tentatively using their selectors a bit, although find I have to retreat to Xpath quite a bit. Playwright seems to be the obvious choice for testing - but automation who knows.

If you can be playwright for automation instead of testing - that'd be a biiiig market. Lot's and lot's of scrapers out there and it's a market microsoft is not interested in.

Re: Show HN: Autotab – An AI-powered Chrome extension to create Selenium scripts

#76

Post by jonasnelle »

omarfarooq wrote: post #73

What would make this maximally useful is if an LLM with a ruleset would be available to parse and act on dynamic situations on pages. For example, add to cart, if the LLM detect it's out of stock (a rule), then do some other action. What's the possibility of LLM-reinforced, rules based branching logic like this being possible with your software in the future?

Couldn’t agree more! Working on this actively. Interesting questions include 1) what format is most effective for the user to convey their intent (maybe not point & click) and 2) how to represent the model output such that it is auditable and editable.

Re: Show HN: Autotab – An AI-powered Chrome extension to create Selenium scripts

#78

Post by omarfarooq »

jonasnelle wrote: post #76
omarfarooq wrote: post #73

What would make this maximally useful is if an LLM with a ruleset would be available to parse and act on dynamic situations on pages. For example, add to cart, if the LLM detect it's out of stock (a rule), then do some other action. What's the possibility of LLM-reinforced, rules based branching logic like this being possible with your software in the future?

Couldn’t agree more! Working on this actively. Interesting questions include 1) what format is most effective for the user to convey their intent (maybe not point & click) and 2) how to represent the model output such that it is auditable and editable.

Awesome to hear, it's already useful now for us to get started with using it, but this kind of evolution would be panacea.

I suggest a Discord group for Autotab to start building up a community! Looking forward.

Re: Show HN: Autotab – An AI-powered Chrome extension to create Selenium scripts

#79

Post by dr_kiszonka »

jonasnelle wrote: post #56
dr_kiszonka wrote: post #54

Hey, it looks very neat. I was wondering if Autotab currently supports logging into websites that require 2FA. (Your readme mentions Google only.)

Hey, we're looking to build out our library of authentication plugins. Many websites offer "Sign in with Google" which makes things easier, you can just specify that in the autotab.yaml file. If you have a specific website in mind, I'd be happy to add support for it!

Thanks for the reply! I was thinking about an internal website. I don't know if this is a reasonable suggestion, but maybe you could focus on common SSOs and 2FA providers like Duo and OneLogin?

Re: Show HN: Autotab – An AI-powered Chrome extension to create Selenium scripts

#80

Post by lucgagan »

I am building _exactly_ the same thing for Playwright over at https://ray.run/. I think this is the future of writing tests no doubt. Planning to launch next week.

By the looks of it, I am taking a slightly different approach than you. I am using LLM only to identify elements, but the actual generators are created using https://ray.run/browser-extension

Post reply on HN