Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Finic – Open source platform for building browser automations

github.com

Post reply on HN
Print view View original

41–50 of 79 posts

Re: Show HN: Finic – Open source platform for building browser automations

#41

Post by thestepafter »

dataviz1000 wrote: post #32
ayanb9440 wrote: post #28

Earlier quoted context omitted.

That's a great suggestion! Essentially a cron job to check for website changes before your automation runs and possibly breaks. What does this check look like for you? Do you just diff the html to see if there are any changes?

The issue with diffing html is selectors are autogenerated with any update to a website's code. Often website which combat scraping will autogenerate different HTML. First thing is to screen caption a website for comparison. Second, it is possible to determine all the visible elements on a page. With Playwright, inject event listeners to all elements on a page and start automated clicking. If the agent fills out form…

Are you doing screenshot comparison with Playwright? If so, how? Based on my research this looks to be a missing feature but I could be incorrect.

Re: Show HN: Finic – Open source platform for building browser automations

#42

Post by krick »

Does anyone know solid (not SaaS, obviously) solution for scraping these days? It's getting pretty hard to get around some pretty harmless cases (like bulk-downloading MY OWN gpx tracks from some fucking fitness-watch servers), with all these js tricks, countless redirects, cloudflare and so on. Even if you already have the cookies, getting non-403 response to any request is very much not trivial. I feel like it's time to upgrade my usual approach of python requests+libxml, but I don't know if there is a library/tool that solves some of the problems for you.

Re: Show HN: Finic – Open source platform for building browser automations

#43

Post by suriya-ganesh »

I've been working on browser agent the last week[1]. So this is very exciting. There are also browser agent implementations like Skyvern[2] (Also YC backed) ,or Tarsier[3] Seems like, finic is providing a way to scale/schedule these agents? If that's the case what's the advantage over something like airflow or windmill ?

If I remember correctly, Skyvern also has an implementation of scaling these browser tasks built in.

ps. Is it not called Robotic Process Automation? First time I'm hearing it as Remote process Automation.

[1]https://github.com/ProductLoft/arachne

[2]https://www.skyvern.com/

[3]https://github.com/reworkd/tarsier

Re: Show HN: Finic – Open source platform for building browser automations

#44

Post by ilrwbwrkhv »

ayanb9440 wrote: post #25
ilrwbwrkhv wrote: post #8

Backed by YC = Not open source. Eventually pressure to exit and hyper scale will take over.

There are quite a few open source YC startups at this point. Our understanding is that: 1. Developer tooling should be open source by default 2. Open source doesn't meaningfully affect revenue/scaling because developers that would use your self-hosted version would build in-house anyway.

I know there are quite a few open source by default companies. But the ethos of open source is sharing / building something by the community and getting paid in a way which does not scale the way VC funding expectations work.

So to have some respect for the open source way on top of which you are building all this please stop advertising it as "open source infrastructure" in bold and sell it like a normal software product with "source available" on the footer.

If you do plan to go open source and actually follow its ethos, remove the funded by VC label and have self hosting front and center in the docs with the hosted bit somewhere in the footer.

Re: Show HN: Finic – Open source platform for building browser automations

#45

Post by ilrwbwrkhv »

ilrwbwrkhv wrote: post #44
ayanb9440 wrote: post #25

Earlier quoted context omitted.

There are quite a few open source YC startups at this point. Our understanding is that: 1. Developer tooling should be open source by default 2. Open source doesn't meaningfully affect revenue/scaling because developers that would use your self-hosted version would build in-house anyway.

I know there are quite a few open source by default companies. But the ethos of open source is sharing / building something by the community and getting paid in a way which does not scale the way VC funding expectations work. So to have some respect for the open source way on top of which you are building all this please stop advertising it as "open source infrastructure" in bold and sell it like a normal software pr…

Like again if you are not sure, what open source means, this is open source: https://appimage.org/

Hope it is abundantly clear with this example. Docker tried it's best to do the whole open source but business first and it led to disastrous results.

At best this will make your company suffer and second guess itself and at worst this is moral fraud.

Talk to your group partner about this and explain to them as well.

Re: Show HN: Finic – Open source platform for building browser automations

#46

Post by ghxst »

jasonwcfan wrote: post #13
ghxst wrote: post #12

Cool service but how will you deal / how do you plan to deal with anti scraping and anti bot services like Akamai, Arkose, Cloudflare, DataDome etc.? Automation of the web isn't solved by another playwright or puppeteer abstraction, you need to solve more fundemental problems in order to mitigate the issues you run into at scale.

I mentioned this in another comment, but I know from experience that it's impossible to reliably differentiate bots from humans over a network. And since the right to automate browsers has survived repeated legal challenges, all vendors can do is make it incrementally harder to weed out the low sophistication actors. This actually creates an evergreen problem that companies need to overcome, and our paid version will…

> but I know from experience that it's impossible to reliably differentiate bots from humans over a network

While this might be true in theory, it doesn't stop them from trying! And believe me, it's getting to a point where the WAF settings on some websites are even annoying the majority of the real users! Some of the issues I am hinting at however are fundemental issues you run into when automating the web using any mainstream browser that hasn't had some source code patches, I'm curious to see if a solution to that will be part of your service if you decide to tackle it.

Re: Show HN: Finic – Open source platform for building browser automations

#47

Post by ghxst »

msp26 wrote: post #38
mdaniel wrote: post #33

> Finic uses Playwright to interact with DOM elements, and recommends BeautifulSoup for HTML parsing. I have never, ever understood anyone who goes to the trouble of booting up a browser, and then uses a python library to do static HTML parsing Anyway, I was surfing around the repo trying to find what, exactly "Safely store and access credentials using Finic’s built-in secret manager" means

What would you recommend for parsing instead?

In python specifically I like lxml (pretty sure that's what BS uses under the hood?), parse5 if you're using node is usually my go to. Ideally though you shouldn't really have to parse anything (or not much at all) when doing browser automation as you have access to the DOM which gives you an interface that accepts query selectors directly (you don't even need the Runtime domain for most of your needs).

Re: Show HN: Finic – Open source platform for building browser automations

#48

Post by sahmeepee »

thestepafter wrote: post #41
dataviz1000 wrote: post #32

Earlier quoted context omitted.

The issue with diffing html is selectors are autogenerated with any update to a website's code. Often website which combat scraping will autogenerate different HTML. First thing is to screen caption a website for comparison. Second, it is possible to determine all the visible elements on a page. With Playwright, inject event listeners to all elements on a page and start automated clicking. If the agent fills out form…

Are you doing screenshot comparison with Playwright? If so, how? Based on my research this looks to be a missing feature but I could be incorrect.

Playwright has screenshot comparison built in, including screenshotting a single element, blanking specific elements, and comparing the textual aspects of elements without a visual comparison. You can even apply a specific stylesheet for comparisons.

Everything I can see in this demo can be done with Playwright on its own or with some very basic infrastructure e.g. from Azure to run the tests (automations). I can't see what it is adding. Is it doing some bot-detection countermeasures?

Checking if the page behaviour has changed is pretty easy in Playwright because its primary purpose is testing, so just write some tests to assert the behaviour you expect before you use it.

We use Playwright to both automate and scrape the site of a public organisation we are obliged to use, as another public body. They do have some bot detection because we get an email when we run the scripts, asking us to confirm our account hasn't been compromised, but so far we have not been blocked. If they ever do block us we will need to hire someone to do manual data entry, but the automation has already paid for itself many times over in a couple of years.

Re: Show HN: Finic – Open source platform for building browser automations

#49

Post by ghxst »

dataviz1000 wrote: post #23

I build browser automation systems with either Playwright or Chrome Extensions. The biggest issue with automating 3rd party websites is knowing when the 3rd party developer pushes changes which break the automation. The way I dealt with that is run a headless browser in the cloud which checks the behavior of the automated site periodically sending emails and sms messages when it breaks. If you don't already have this…

IO between humans and websites can be broken down to only a few fundamental pieces (or elements I should say). This is actually where AI has a lot of opportunity to add value as it has the capability of significantly reducing the possibilty of breakage between changes.

Re: Show HN: Finic – Open source platform for building browser automations

#50

Post by ghxst »

jasonwcfan wrote: post #5
Oras wrote: post #4

Don't take this as a negative thing, but I'm confused. Is it a playwright? Is it a residential proxy? It's not clear from your video.

Proxies are definitely on our roadmap, but for now it just supports stock Playwright. Thanks for the feedback! I just updated the repo to make it more clear that it's Playwright based. Once my cofounder wakes up I'll see if he can re-record the video as well.

What kind of proxies are on your road map, do you have any experience with in-house proxy networks?
Post reply on HN