Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Finic – Open source platform for building browser automations

github.com

Post reply on HN
Print view View original

31–40 of 79 posts

Re: Show HN: Finic – Open source platform for building browser automations

#31

Post by mdaniel »

ayanb9440 wrote: post #19
slewis wrote: post #15

Is it stateful? Like can I do a run, read the results, and then do another run from that point?

We currently don't save the browser state after the run has completed but that's something we can definitely add as a feature. Could you elaborate on your use case? In which scenarios would it be better to split a run into multiple steps?

Almost any process that involves the word "workflow" (my mental model is one where the user would press alt-tab to look up something else in another window). The very, very common case would be one where they have a stupid SMS-based or "click email link" login flow: one would not wish to do that a ton, versus just leaving the session authenticated for reuse later in the day

Also, if my mental model is correct, the more browsing and mouse-movement telemetry those cloudflare/akamai/etc gizmos encounter, the more likely they are to think the browser is for real, versus encountering a "fresh" one is almost certainly red-alert. Not a panacea, for sure, but I'd guess every little bit helps

Re: Show HN: Finic – Open source platform for building browser automations

#32

Post by dataviz1000 »

ayanb9440 wrote: post #28
dataviz1000 wrote: post #23

I build browser automation systems with either Playwright or Chrome Extensions. The biggest issue with automating 3rd party websites is knowing when the 3rd party developer pushes changes which break the automation. The way I dealt with that is run a headless browser in the cloud which checks the behavior of the automated site periodically sending emails and sms messages when it breaks. If you don't already have this…

That's a great suggestion! Essentially a cron job to check for website changes before your automation runs and possibly breaks. What does this check look like for you? Do you just diff the html to see if there are any changes?

The issue with diffing html is selectors are autogenerated with any update to a website's code. Often website which combat scraping will autogenerate different HTML. First thing is to screen caption a website for comparison. Second, it is possible to determine all the visible elements on a page. With Playwright, inject event listeners to all elements on a page and start automated clicking. If the agent fills out forms, then make sure that all fields are available to populate. There are a lot of heuristics.

Re: Show HN: Finic – Open source platform for building browser automations

#33

Post by mdaniel »

> Finic uses Playwright to interact with DOM elements, and recommends BeautifulSoup for HTML parsing.

I have never, ever understood anyone who goes to the trouble of booting up a browser, and then uses a python library to do static HTML parsing

Anyway, I was surfing around the repo trying to find what, exactly "Safely store and access credentials using Finic’s built-in secret manager" means

Re: Show HN: Finic – Open source platform for building browser automations

#34

Post by rgrieselhuber »

tempest_ wrote: post #27
jasonwcfan wrote: post #21

Earlier quoted context omitted.

Why is it unethical when courts have repeatedly affirmed browser automation to be legal and permitted? If anything, it's unethical for companies to dictate how their customers can access services they've already paid for. If I'm paying hundreds of thousands per year for software, shouldn't I be allowed to build automations over it? Instead, many enterprise products go to great lengths to restrict this kind of usage.…

> they've already paid for. That is the crux, rarely is it a service being scraped that they paid for

Many times it is scraping aggregators of data that those aggregators also did not pay for.

Re: Show HN: Finic – Open source platform for building browser automations

#35

Post by ayanb9440 »

mdaniel wrote: post #33

> Finic uses Playwright to interact with DOM elements, and recommends BeautifulSoup for HTML parsing. I have never, ever understood anyone who goes to the trouble of booting up a browser, and then uses a python library to do static HTML parsing Anyway, I was surfing around the repo trying to find what, exactly "Safely store and access credentials using Finic’s built-in secret manager" means

We're in the middle of putting this together right now but it's going to be a wrapper around Google Secret Manager for those that don't want to set up a secrets manager themselves.

Re: Show HN: Finic – Open source platform for building browser automations

#36

Post by jasonwcfan »

mdaniel wrote: post #31
ayanb9440 wrote: post #19

Earlier quoted context omitted.

We currently don't save the browser state after the run has completed but that's something we can definitely add as a feature. Could you elaborate on your use case? In which scenarios would it be better to split a run into multiple steps?

Almost any process that involves the word "workflow" (my mental model is one where the user would press alt-tab to look up something else in another window). The very, very common case would be one where they have a stupid SMS-based or "click email link" login flow: one would not wish to do that a ton, versus just leaving the session authenticated for reuse later in the day Also, if my mental model is correct, the mo…

The way we plan to handle authenticated sessions is through a secret management service with the ability to ping an endpoint to check if the session is still valid, and if not, run a separate automation that re-authenticates and updates the secret manager with the new token. In that case, it wouldn't need to be stateful, but I can certainly see a case for statefulness being useful as workflows get even more complex.

As for device telemetry, my experience has been that most companies don't rely too much on it. Any heuristic used to identify bots is likely to have a high false positive rate and include many legitimate users, who then complain about it. Captchas are much more common and effective, though if you've seen some of the newer puzzles that vendors like Arkose Labs offers, it's a tossup whether the median human intelligence can even solve it.

Re: Show HN: Finic – Open source platform for building browser automations

#37

Post by 0x3444ac53 »

mdaniel wrote: post #33

> Finic uses Playwright to interact with DOM elements, and recommends BeautifulSoup for HTML parsing. I have never, ever understood anyone who goes to the trouble of booting up a browser, and then uses a python library to do static HTML parsing Anyway, I was surfing around the repo trying to find what, exactly "Safely store and access credentials using Finic’s built-in secret manager" means

Often times websites won't load the HTML without executing the JavaScript. or uses JavaScript running client side to generate the entire page.

Re: Show HN: Finic – Open source platform for building browser automations

#38

Post by msp26 »

mdaniel wrote: post #33

> Finic uses Playwright to interact with DOM elements, and recommends BeautifulSoup for HTML parsing. I have never, ever understood anyone who goes to the trouble of booting up a browser, and then uses a python library to do static HTML parsing Anyway, I was surfing around the repo trying to find what, exactly "Safely store and access credentials using Finic’s built-in secret manager" means

What would you recommend for parsing instead?

Re: Show HN: Finic – Open source platform for building browser automations

#39

Post by mdaniel »

msp26 wrote: post #38
mdaniel wrote: post #33

> Finic uses Playwright to interact with DOM elements, and recommends BeautifulSoup for HTML parsing. I have never, ever understood anyone who goes to the trouble of booting up a browser, and then uses a python library to do static HTML parsing Anyway, I was surfing around the repo trying to find what, exactly "Safely store and access credentials using Finic’s built-in secret manager" means

What would you recommend for parsing instead?

In this specific scenario, where the project is using *automated Chrome* to even bother with the connection, redirects, and bazillions of other "browser-y" things to arrive at HTML to be parsed, the very idea that one would `soup = BeautifulSoup(playright.content())` is crazypants to me

I am open to the fact that html5lib strives to parse correctly, and good for them, but that would be the case where one wished to use python for parsing to avoid the pitfalls of dragging a native binary around with you

Re: Show HN: Finic – Open source platform for building browser automations

#40

Post by mdaniel »

0x3444ac53 wrote: post #37
mdaniel wrote: post #33

> Finic uses Playwright to interact with DOM elements, and recommends BeautifulSoup for HTML parsing. I have never, ever understood anyone who goes to the trouble of booting up a browser, and then uses a python library to do static HTML parsing Anyway, I was surfing around the repo trying to find what, exactly "Safely store and access credentials using Finic’s built-in secret manager" means

Often times websites won't load the HTML without executing the JavaScript. or uses JavaScript running client side to generate the entire page.

I feel that we are in agreement for the cases where one would use Playwright, and for damn sure would not involve BS4 for anything in that case
Post reply on HN