Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: An API that takes a URL and returns a file with browser screenshots

github.com

Post reply on HN
Print view View original

81–90 of 107 posts

Re: Show HN: An API that takes a URL and returns a file with browser screenshots

#81

Post by pavel_lishin »

ranger_danger wrote: post #67
cmgriffing wrote: post #33

Earlier quoted context omitted.

Quick note: when trying to do full page screenshots, Chrome does a screenshot of the current view, then scrolls and does another screenshot. This can cause some interesting artifacts when rendering pages with scroll behaviors. Firefox does a proper full page screenshot and even allows you to set a higher DPS value. I use this a lot when making video content. Check out some of the args in FF using: `:screenshot --help…

Where would you type that command in?

Firefox developer console.

Re: Show HN: An API that takes a URL and returns a file with browser screenshots

#82

Post by sgerenser »

gkamer8 wrote: post #16
bangaladore wrote: post #12

Earlier quoted context omitted.

The website [1] is very strange. What does U.S. stand for? If I were to stumble on this I'd assume it was a fishing / scam website trying to impersonate the government. Bad vibes all around. [1] - https://us.ai/

I'll try to improve the vibes :( I've been working at this startup for almost two years now and that page and branding etc has been changing a lot as you can imagine ...

Props for using Garamond Condensed and giving me flashbacks to 1990s Apple.

Re: Show HN: An API that takes a URL and returns a file with browser screenshots

#83

Post by codenote »

I thought it was a scale of code that could have been included in Abbe. https://github.com/US-Artificial-Intelligence/abbey

Was the motivation for separating it based on security considerations, as stated in the "Security Considerations"? https://github.com/US-Artificial-Intelligence/ScrapeServ?tab...

Re: Show HN: An API that takes a URL and returns a file with browser screenshots

#84

Post by gkamer8 »

codenote wrote: post #83

I thought it was a scale of code that could have been included in Abbe. https://github.com/US-Artificial-Intelligence/abbey Was the motivation for separating it based on security considerations, as stated in the "Security Considerations"? https://github.com/US-Artificial-Intelligence/ScrapeServ?tab...

Yes, sort of - that and scaling reasons. It's actually in that same repo now but in a different service. I'd like to remove it from the Abbey repo entirely eventually.

Re: Show HN: An API that takes a URL and returns a file with browser screenshots

#85

Post by jot »

ALittleLight wrote: post #61
jot wrote: post #59

Earlier quoted context omitted.

I’m sure we can do better here. In my experience our customers are more worried about having the service stop when they hit the limit of a tier than they are about being charged a few more dollars.

Maybe I'm misreading. It sounds like you're stepping the user up a pricing tier - e.g. going from 50 a month to 100 and then charging at the better rate. I would also worry about a bug on my end that fires off lots of screenshots. I would expect a quota or limit to protect me from that.

That’s right. On our standard self-service plans we automatically charge a better rate as volume increases. You only pay the difference between tiers as you move through them.

It’s rare that anyone makes that kind of mistake. It probably helps that our rate limits are relatively low compared to other APIs and we email you when you get close to stepping up a tier. If you did make such a mistake we would, like all good dev tools, work with you to resolve. If it happened a lot we might introduce some additional controls.

We’ve been in this business for over 12 years and currently have over 700 customers so we’re fairly confident we have the balance right.

Re: Show HN: An API that takes a URL and returns a file with browser screenshots

#86

Post by jot »

jchw wrote: post #70

One thing to be cognizant of: if you're planning to run this sort of thing against potentially untrusted URLs, the browser might be able to make requests to internal hosts in whatever network it is on. It would be wise, on Linux, to use network namespaces, and block any local IP range in the namespace, or use a network namespace to limit the browser to a wireguard VPN tunnel to some other network.

Too many developers learn this the hard way.

It’s one of the top reasons larger organisations prefer to use hosted services rather than doing it themselves.

Re: Show HN: An API that takes a URL and returns a file with browser screenshots

#87

Post by leptons »

jchw wrote: post #70

One thing to be cognizant of: if you're planning to run this sort of thing against potentially untrusted URLs, the browser might be able to make requests to internal hosts in whatever network it is on. It would be wise, on Linux, to use network namespaces, and block any local IP range in the namespace, or use a network namespace to limit the browser to a wireguard VPN tunnel to some other network.

This is true for practically every web browser anyone uses on any site that they don't personally control.

Re: Show HN: An API that takes a URL and returns a file with browser screenshots

#88

Post by arnoldcjones »

kevinsundar wrote: post #53

I'm looking for something similar that can also extract the diff of content on the page over time, in addition to screenshots. Any suggestions? I have a homegrown solution using an LLM and scrapegraphai for https://getchangelog.com but would rather offload that to a service that does a better job rendering websites. There's some websites that I get error pages from using playwright, but they load fine in my usual Chr…

Good point on offloading it as for the amount of work that's required in setting up a wrapper for something like Puppeteer, Playwright etc that also works with a probably quite specific setup, I've found the best way to get a quality image consistently is to just subscribe to one of the many SASS' out there that already do this well. Some of the comments above suggest some decent screenshot-as-a-service products.

Really depends on how valuable your time is over your (or your companies) money. I prefer going for the quality (and more $) solution rather than the solution that boasts cheap prices, as I tend to avoid headaches of unreliable services. Sam Vines Boots theory and all that.

For image comparison I've always found using pixelmatch by Mapbox works well for PNG's

https://github.com/mapbox/pixelmatch

Re: Show HN: An API that takes a URL and returns a file with browser screenshots

#90

Post by genewitch »

wereHamster wrote: post #50
cmgriffing wrote: post #33

Earlier quoted context omitted.

Quick note: when trying to do full page screenshots, Chrome does a screenshot of the current view, then scrolls and does another screenshot. This can cause some interesting artifacts when rendering pages with scroll behaviors. Firefox does a proper full page screenshot and even allows you to set a higher DPS value. I use this a lot when making video content. Check out some of the args in FF using: `:screenshot --help…

That's not the behavior I'm seeing (with Puppeteer). Any elements positioned relative to the viewport stay within the area specified by screen size (eg. 1200x800) which is usually the top of the page. If the browser would scroll down these would also move down (and potentially appear multiple times in the image). Also intersection observers which are further down on the page do not trigger when I do a full-page scree…

bravo for puppeteer, i guess? "singlefile" is the only thing i've ever seen not do weird artifacts in the middle of some site renders, or, like on reddit, just give up rendering comments and render blank space instead until the footer.

anyhow i've been doing this exact thing for a real long time, e.g.

https://raw.githubusercontent.com/genewitch/opensource/refs/...

using bash to return json to some stupid chat service we were running

Post reply on HN