Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Autotab – An AI-powered Chrome extension to create Selenium scripts

autotab.com

Post reply on HN
Print view View original

121–130 of 131 posts

Re: Show HN: Autotab – An AI-powered Chrome extension to create Selenium scripts

#121

Post by jonasnelle »

eshack94 wrote: post #117

This is super cool! What are some use cases that you recommend this for, outside of testing browser workflows?

We’re actually mainly focused on automations more than testing. We want to build autotab into your intern/assistant for browser-based tasks that you would rather have someone else do.

For example, autotab helped a designer automate client handoffs. He set up an automation that exported assets from Figma and uploaded them to Google Drive, updated his CRM and notified the client.

Re: Show HN: Autotab – An AI-powered Chrome extension to create Selenium scripts

#122

Post by jonasnelle »

clay09 wrote: post #120

Nice work! What are some differences between this project and Katalon Recorder? Katalon Recorder is also a chrome extension, allows recording of actions, and export to Python.

Katalon looks like a cool project, hadn’t heard of them before! We’re more focused on automations as opposed to testing, which is most important for where we take the product next. More focus for us on handling tasks that are less deterministic/require a bit more intelligence.

Re: Show HN: Autotab – An AI-powered Chrome extension to create Selenium scripts

#123

Post by mthoms »

jonasnelle wrote: post #113
mthoms wrote: post #112

Hopefully you're still monitoring this page: Do the HTML contents of the page get sent to your server for processing? What guarantees are there over privacy? One of the uses I see for this is in helping users scrape their own transactions from banking and other financial websites.

Yes, we process the contents of the page when users are recording on our servers but don’t store the page HTML. I don’t see any world in which we would sell data, but in case you’re worried about the security of our servers I’m not sure there’s much I can say other than that we use standard security best practices including SSL, 2FA on cloud accounts etc. Does that answer your question?

It does, thanks.

Re: Show HN: Autotab – An AI-powered Chrome extension to create Selenium scripts

#125

Post by brianjking »

lucgagan wrote: post #80

I am building _exactly_ the same thing for Playwright over at https://ray.run/ . I think this is the future of writing tests no doubt. Planning to launch next week. By the looks of it, I am taking a slightly different approach than you. I am using LLM only to identify elements, but the actual generators are created using https://ray.run/browser-extension

Ooh, ty! Going to ping you about this.

Re: Show HN: Autotab – An AI-powered Chrome extension to create Selenium scripts

#126

Post by alexirobbins »

omarfarooq wrote: post #78
jonasnelle wrote: post #76

Earlier quoted context omitted.

Couldn’t agree more! Working on this actively. Interesting questions include 1) what format is most effective for the user to convey their intent (maybe not point & click) and 2) how to represent the model output such that it is auditable and editable.

Awesome to hear, it's already useful now for us to get started with using it, but this kind of evolution would be panacea. I suggest a Discord group for Autotab to start building up a community! Looking forward.

We just setup a discord! https://discord.gg/seGGxSUgzM

Re: Show HN: Autotab – An AI-powered Chrome extension to create Selenium scripts

#127

Post by lobochrome »

jonasnelle wrote: post #83
lobochrome wrote: post #75

Earlier quoted context omitted.

Yeah - I moved my entire automation library over at the beginning of the year. The docker integrations and ease of switching out browsers without having to manage webdrivers individually sealed the deal. I'm also tentatively using their selectors a bit, although find I have to retreat to Xpath quite a bit. Playwright seems to be the obvious choice for testing - but automation who knows. If you can be playwright for a…

Interesting, can you say more about why you prefer Playwright’s selectors? Helpful to have some anecdotes from real experience!

I don’t prefer I’m just playing. If they work, the readability of the code is just simpler vs xpath.

But it’s less than 5% of selectors across the code base

Re: Show HN: Autotab – An AI-powered Chrome extension to create Selenium scripts

#128

Post by eastendguy »

jonasnelle wrote: post #25
hugs wrote: post #7

Selenium project creator here... Very cool! I totally expect testing to be one of the killer apps for AI. And this is an easy way to get there. How do you ensure the AI isn't hallucinating any details in the generated code?

That's awesome! We couldn't have built any of this without Selenium, definitely standing on the shoulders of giants :) Right now we have the basic sanity check that the Selenium xpath selects exactly one element in the DOM. Going forward I think the best way to do this is to have a live preview where you can see the mirrored browser copy your last action and debug if it errors or isn't what you wanted.

> and debug if it errors or isn't what you wanted.

That is a potential problem Because at this point the user will still need good HTML/Selenium/coding knowledge to debug. We ran into the same issue using chatgpt generated scripts for ui.vision. Once a QA person has to figure out why some generated code does not work, it becomes a hassle and removes any initial advantage over the classic record & replay approach.

Re: Show HN: Autotab – An AI-powered Chrome extension to create Selenium scripts

#129

Post by jonasnelle »

eastendguy wrote: post #128
jonasnelle wrote: post #25

Earlier quoted context omitted.

That's awesome! We couldn't have built any of this without Selenium, definitely standing on the shoulders of giants :) Right now we have the basic sanity check that the Selenium xpath selects exactly one element in the DOM. Going forward I think the best way to do this is to have a live preview where you can see the mirrored browser copy your last action and debug if it errors or isn't what you wanted.

> and debug if it errors or isn't what you wanted. That is a potential problem Because at this point the user will still need good HTML/Selenium/coding knowledge to debug. We ran into the same issue using chatgpt generated scripts for ui.vision. Once a QA person has to figure out why some generated code does not work, it becomes a hassle and removes any initial advantage over the classic record & replay approach.

I agree, to really debug they need coding knowledge today.

But there's something in between where you can say "try again" or even give high level feedback like "don't click the element with that exact title, just click whichever one is first in the list". My working hypothesis is that these are bigger than the ones where you need coding knowledge. Certainly this category will only grow as language models become better.

Re: Show HN: Autotab – An AI-powered Chrome extension to create Selenium scripts

#130

Post by jonasnelle »

btbuildem wrote: post #77

> This will launch a Chrome session controlled by Selenium and then log you in to Google Why is logging into Google a requirement?

Fyi we fixed this so now you can get an API key at autotab.com/dashboard and then that is sufficient to authenticate when you run `autotab record`, no more logging in to Google in the Selenium browser!
Post reply on HN