Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: An API that takes a URL and returns a file with browser screenshots

github.com

Post reply on HN
Print view View original

71–80 of 107 posts

Re: Show HN: An API that takes a URL and returns a file with browser screenshots

#73

Post by [deleted] »

cmgriffing wrote: post #33
xnx wrote: post #4

For anyone who might not be aware, Chrome also has the ability to save screenshots from the command line using: chrome --headless --screenshot="path/to/save/screenshot.png" --disable-gpu --window-size=1280,720 " https://www.example.com "

Quick note: when trying to do full page screenshots, Chrome does a screenshot of the current view, then scrolls and does another screenshot. This can cause some interesting artifacts when rendering pages with scroll behaviors. Firefox does a proper full page screenshot and even allows you to set a higher DPS value. I use this a lot when making video content. Check out some of the args in FF using: `:screenshot --help…

[deleted]

Re: Show HN: An API that takes a URL and returns a file with browser screenshots

#74

Post by [deleted] »

wereHamster wrote: post #50
cmgriffing wrote: post #33

Earlier quoted context omitted.

Quick note: when trying to do full page screenshots, Chrome does a screenshot of the current view, then scrolls and does another screenshot. This can cause some interesting artifacts when rendering pages with scroll behaviors. Firefox does a proper full page screenshot and even allows you to set a higher DPS value. I use this a lot when making video content. Check out some of the args in FF using: `:screenshot --help…

That's not the behavior I'm seeing (with Puppeteer). Any elements positioned relative to the viewport stay within the area specified by screen size (eg. 1200x800) which is usually the top of the page. If the browser would scroll down these would also move down (and potentially appear multiple times in the image). Also intersection observers which are further down on the page do not trigger when I do a full-page scree…

[deleted]

Re: Show HN: An API that takes a URL and returns a file with browser screenshots

#75

Post by remram »

jchw wrote: post #70

One thing to be cognizant of: if you're planning to run this sort of thing against potentially untrusted URLs, the browser might be able to make requests to internal hosts in whatever network it is on. It would be wise, on Linux, to use network namespaces, and block any local IP range in the namespace, or use a network namespace to limit the browser to a wireguard VPN tunnel to some other network.

Very important note! This is called Server-Side Request Forgery (SSRF).

Re: Show HN: An API that takes a URL and returns a file with browser screenshots

#76

Post by hedora »

It'd be nice if it produced a list of bounding boxes + URL's you'd get if you clicked on the bounding box.

Then it'd be close to my dream of a serverless web browser service, where the client just renders a clickmap .png or .webp, and the requests go to a farm of "one request per page load" ephemeral web browser instances. The web browsers could cache the images + clickmaps they return in an S3 bucket.

Assuming the farm of browsers had a large number of users, this would completely defeat fingerprinting + cookies. It'd also provide an archive (as in durable, not as in high quality) of the browsed static content.

Re: Show HN: An API that takes a URL and returns a file with browser screenshots

#77

Post by tonyhart7 »

basilgohar wrote: post #47
bbor wrote: post #45

Earlier quoted context omitted.

Do these services respect norobot manifests? Isn't this all kinda... illegal...? Or at least non-consensual?

robots.txt isn't legally binding. I am interested to know if and how services even interact with it. It's more like a clue on when the interesting content for scrapers is on your site. This is how I imagine it goes: "Hey, don't scrape the data here." "You know what? I'm scrape it even harder!"

it is legally binding if your company based on SV (only California implement this law) and they can prove it

Re: Show HN: An API that takes a URL and returns a file with browser screenshots

#79

Post by yjftsjthsd-h »

blueflow wrote: post #41
input_sh wrote: post #34

Earlier quoted context omitted.

Firefox equivalent: firefox -screenshot file.png https://example.com --window-size=1280,720 A bit annoyingly, it won't work if you have Firefox already open.

> it won't work if you have Firefox already open now try and go ahead how you could isolate these instances so they cannot see each other. this leads into a rabbit hole of bad design.

> now try and go ahead how you could isolate these instances so they cannot see each other. this leads into a rabbit hole of bad design.

Okay, done:

  PROFILEDIR="$(mktemp -d)"
  firefox --no-remote  --profile "$PROFILEDIR" --screenshot $PWD/output.png https://xkcd.com
  rm -r "$PROFILEDIR"
What's the rabbit hole?
Post reply on HN