Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Puppeteer Support for Firefox

hacks.mozilla.org

Post reply on HN
Print view View original

51–60 of 92 posts

Re: Puppeteer Support for Firefox

#51

Post by jesprenj »

What I very dislike about current browser automation tools is that they all use TCP for connecting the browser with the manager program. This means that, unlike for UNIX domain sockets, filesystem permissions (user/group restrictions) cannot be used to protect the TCP socket, which opens the browser automation ecosystem to many attacks where 127.0.0.1 cannot be trusted (untrusted users on a shared host).

I have yet to see a browser automation tool that does not use localhost bound TCP sockets. Apart from that, most tools do not offer strong authentication -- a browser is spawned and it listens on a socket and when the controlling application connects to the browser management socket, no authentication is required by default, which creates hidden vulnerabilites.

While browser sessions may only be controlled by knowing their random UUIDs, creating new sessions is usually possible to anyone on 127.0.0.1.

I don't know really, it's quite possible I'm just spreading lies here, please correct me and expand on this topic a bit.

Re: Puppeteer Support for Firefox

#52

Post by JoelEinbinder »

jesprenj wrote: post #51

What I very dislike about current browser automation tools is that they all use TCP for connecting the browser with the manager program. This means that, unlike for UNIX domain sockets, filesystem permissions (user/group restrictions) cannot be used to protect the TCP socket, which opens the browser automation ecosystem to many attacks where 127.0.0.1 cannot be trusted (untrusted users on a shared host). I have yet t…

You can set `pipe` to true in puppeteer (default false) here https://pptr.dev/api/puppeteer.launchoptions

By default, Playwright launches this way and you have to specifically enable the tcp listening.

Re: Puppeteer Support for Firefox

#53

Post by localfirst »

hugs wrote: post #42
localfirst wrote: post #15

Earlier quoted context omitted.

interesting so you are emulating hardware inputs from RPi how is it reading whats on the screen? computer vision?

Not to make this an ad for my project, but I'm starting to document it more here: https://valetnet.dev/ The Raspberry Pi is configured to use the USB HID protocol to look and act like a mouse and keyboard when plugged into a phone. (Android and iOS now support mouse and keyboard inputs). For video, we have two models: - "Valet Link" uses an HDMI capture card (and a multi-port dongle) to pull the video signal directly…

> Could probably write a book about the lessons learned getting a "vision first" approach to automation working

ha that would be splendid! please do maybe even a blog on valetnet.dev (lovely site btw a demo or video would be a nice)

I'm convinced vision first is the way to go despite people saying its slow the benefits are tremendous as lot of websites simply do not play nice with HTML and I do not like having to inspect XHR to figure out APIs

SikuliX was my last love affair with this approach but eventually I lost interest in scraping and automation so I'm pleased to see people still working on vision first automation approaches.

Re: Puppeteer Support for Firefox

#54

Post by ak217 »

mook wrote: post #34
freedomben wrote: post #18

Earlier quoted context omitted.

Indeed, Firefox uses PDF.js which I've found to produce really good results.

Doesn't PDF.js go the other way (convert a PDF into HTML-and-friends for display in a browser, instead of "printing" a page into a PDF)? I haven't dug into it and am quite possibly incorrect, hence the request for confirmation!

That is correct, pdfjs is not usable for printing. Chrome uses Skia for printing, not sure what Firefox uses.

Re: Puppeteer Support for Firefox

#55

Post by hugs »

notinmykernel wrote: post #50
huy-nguyen wrote: post #21

Earlier quoted context omitted.

What’s the relationship between Selenium, Puppeteer and Webdriver BiDi? I’m a happy user of Playwright. Is there any reason why I should consider Selenium or Puppeteer?

I am an active user of both Selenium and Puppeteer/Pyppeteer. I use them because it's what I learned and they still work great, and explicitly because it's not Microsoft.

There are dozens of us... DOZENS!

(Actually, millions... but you wouldn't know it if all you read were comments on HN and Reddit.)

Re: Puppeteer Support for Firefox

#56

Post by _heimdall »

jesprenj wrote: post #51

What I very dislike about current browser automation tools is that they all use TCP for connecting the browser with the manager program. This means that, unlike for UNIX domain sockets, filesystem permissions (user/group restrictions) cannot be used to protect the TCP socket, which opens the browser automation ecosystem to many attacks where 127.0.0.1 cannot be trusted (untrusted users on a shared host). I have yet t…

I have always wanted a browser automation tool that taps directly into the accessibility tree. Plenty do supporting querying based on accessibility features, but unless I'm mistaken none go directly to the same underlying accessibility tree used by screen readers and similar.

Happy to be wrong here if anyone can correct me. The idea of all tests confirming both functionality and accessibility in one go would be much nicer than testing against hard coded test IDs and separately writing a few a11y tests if I'm offered the time.

Re: Puppeteer Support for Firefox

#57

Post by pauldino »

sorenjan wrote: post #36
djbusby wrote: post #32

Earlier quoted context omitted.

I have dreamed about a swappable engine. Like, a wrapper that does my history and tabs and book marks - but let's me move from rendering in Chrome or Gecko or Servo or whatever.

There used to be an extension for Firefox called "IE Tab for Firefox" that used the IE rendering engine inside a Firefox tab, for sites that only worked in IE.

And Google made basically the opposite thing to embed Chrome within Internet Explorer for sites that wouldn't work in IE.

https://en.wikipedia.org/wiki/Google_Chrome_Frame

Re: Puppeteer Support for Firefox

#58

Post by hugs »

localfirst wrote: post #53
hugs wrote: post #42

Earlier quoted context omitted.

Not to make this an ad for my project, but I'm starting to document it more here: https://valetnet.dev/ The Raspberry Pi is configured to use the USB HID protocol to look and act like a mouse and keyboard when plugged into a phone. (Android and iOS now support mouse and keyboard inputs). For video, we have two models: - "Valet Link" uses an HDMI capture card (and a multi-port dongle) to pull the video signal directly…

> Could probably write a book about the lessons learned getting a "vision first" approach to automation working ha that would be splendid! please do maybe even a blog on valetnet.dev (lovely site btw a demo or video would be a nice) I'm convinced vision first is the way to go despite people saying its slow the benefits are tremendous as lot of websites simply do not play nice with HTML and I do not like having to ins…

Agreed on the need for a demo. #1 on the TODO list! If I know at least one person will read it, I might even do a blog, too! :)

The rise of multi-modal LLMs is making "vision first" plausible. However, my basic test is asking these models to find the X,Y screen coordinates of the number "1" on a screenshot of a calculator app. ChatGPT-4o still can't do it. Same with LLaVA 1.5 last I tried. But I'm sure it'll get there someday soon.

Yeah, SikuliX was dependent on old school "classic" OpenCV methods. No machine learning involved. To some extent those methods still work in highly constrained domains like UI automation... But I'm looking forward to sprinkling in some AI magic when it's ready.

Re: Puppeteer Support for Firefox

#59

Post by jahewson »

_heimdall wrote: post #56
jesprenj wrote: post #51

What I very dislike about current browser automation tools is that they all use TCP for connecting the browser with the manager program. This means that, unlike for UNIX domain sockets, filesystem permissions (user/group restrictions) cannot be used to protect the TCP socket, which opens the browser automation ecosystem to many attacks where 127.0.0.1 cannot be trusted (untrusted users on a shared host). I have yet t…

I have always wanted a browser automation tool that taps directly into the accessibility tree. Plenty do supporting querying based on accessibility features, but unless I'm mistaken none go directly to the same underlying accessibility tree used by screen readers and similar. Happy to be wrong here if anyone can correct me. The idea of all tests confirming both functionality and accessibility in one go would be much…

It depends on what you’re testing. Much of a typical page is visual noise that is invisible to the accessibility tree but is often still something you’ll want tests for. It’s also not uncommon for accessible ui paths to differ from regular ones via invisible screen-reader only content, eg in a complex dropdown list. So you can end up with a situation where you test that accessible path works but not regular clicks!

If you really want gold standard screen reader testing, there’s no substitute for testing with actual screen readers. Each uses the accessibility tree in its own way. Remember also that each browser has its own accessibility tree.

Re: Puppeteer Support for Firefox

#60

Post by bryanrasmussen »

jesprenj wrote: post #51

What I very dislike about current browser automation tools is that they all use TCP for connecting the browser with the manager program. This means that, unlike for UNIX domain sockets, filesystem permissions (user/group restrictions) cannot be used to protect the TCP socket, which opens the browser automation ecosystem to many attacks where 127.0.0.1 cannot be trusted (untrusted users on a shared host). I have yet t…

I haven't researched it but I would be surprised if Sikuli does this http://sikulix.com/
Post reply on HN