Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: An API that takes a URL and returns a file with browser screenshots

github.com

Post reply on HN
Print view View original

91–100 of 107 posts

Re: Show HN: An API that takes a URL and returns a file with browser screenshots

#91

Post by genewitch »

input_sh wrote: post #34
xnx wrote: post #4

For anyone who might not be aware, Chrome also has the ability to save screenshots from the command line using: chrome --headless --screenshot="path/to/save/screenshot.png" --disable-gpu --window-size=1280,720 " https://www.example.com "

Firefox equivalent: firefox -screenshot file.png https://example.com --window-size=1280,720 A bit annoyingly, it won't work if you have Firefox already open.

on my firefox if i right click on a part of the page the website hasn't hijacked, it gives the option to "take screenshot" - which i think required enabling a setting somewhere. I hope it wasn't in about:config or wherever the dark-art settings are. I use that feature of FF to screenshot youtube videos with the subtitles moved and the scrub bar cropped out, i feel like it's a cleaner and smaller clipboard copy than using win+shift+s. Microsoft changed a lot about how windows handles ... files ... internally and screenshots are huge .png now, making me miss the days of huge .bmp.

also as mentioned above, if you need entire sites backed up the firefox extension "singlefile" is the business. if image-y things? bulk image downloader (costs money but 100% worth; you know it if you need it: BID); and yt-dlp + ffmpeg for video, in powershell (get 7.5.0 do yourself a favor!)

```powershell

$userInput = Read-Host -Prompt '480 video download script enter URL'

Write-Output "URL:`t`t$userInput"

c:\opt\yt-dlp.exe `

-f 'bestvideo[height--write-auto-subs --write-subs `

--fragment-retries infinite `

$userInput

```

Re: Show HN: An API that takes a URL and returns a file with browser screenshots

#92

Post by blueflow »

yjftsjthsd-h wrote: post #79
blueflow wrote: post #41

Earlier quoted context omitted.

> it won't work if you have Firefox already open now try and go ahead how you could isolate these instances so they cannot see each other. this leads into a rabbit hole of bad design.

> now try and go ahead how you could isolate these instances so they cannot see each other. this leads into a rabbit hole of bad design. Okay, done: PROFILEDIR="$(mktemp -d)" firefox --no-remote --profile "$PROFILEDIR" --screenshot $PWD/output.png https://xkcd.com rm -r "$PROFILEDIR" What's the rabbit hole?

Whats with the dbus interface?

Re: Show HN: An API that takes a URL and returns a file with browser screenshots

#93

Post by jchw »

leptons wrote: post #87
jchw wrote: post #70

One thing to be cognizant of: if you're planning to run this sort of thing against potentially untrusted URLs, the browser might be able to make requests to internal hosts in whatever network it is on. It would be wise, on Linux, to use network namespaces, and block any local IP range in the namespace, or use a network namespace to limit the browser to a wireguard VPN tunnel to some other network.

This is true for practically every web browser anyone uses on any site that they don't personally control.

This is true, although I think in a home environment, there aren't as many interesting things to hit, and you're limited by Same Origin Policy, as well as certain mitigations that web browsers deploy against attacks like DNS Rebinding. However, if you're running this on a server, there's a much greater likelihood that interesting services are under the firewall, e.g. maybe the Kubernetes API server. Code execution could potentially be a form post away.

Re: Show HN: An API that takes a URL and returns a file with browser screenshots

#94

Post by xeornet »

rustdeveloper wrote: post #36
jot wrote: post #26

If you’re worried about the security risks, edge cases, maintenance pain and scaling challenges of self hosting there are various solid hosted alternatives: - https://browserless.io - low level browser control - https://scrapingbee.com - scraping specialists - https://urlbox.com - screenshot specialists* They’re all profitable and have been around for years so you can depend on the businesses and the tech. * Disclosu…

Happy to suggest another web scraping API alternative I rely on: https://scrapingfish.com

What’s the chance you’re affiliated? Almost every one of your comments links to it. And curiously similar interest in Rust from the official HN page and yours. No need to be sneaky.

Re: Show HN: An API that takes a URL and returns a file with browser screenshots

#95

Post by standardly »

throwaway314155 wrote: post #62
wildzzz wrote: post #52

Earlier quoted context omitted.

Thanks, I'm always on the lookout for people with suspicious or over-exaggerated credentials cough-Lex Friedman-cough . Is the national security paper public? Is it something about Ufimstev?

> cough-Lex Friedman-cough Oh please elaborate!

listen to any episode and it's evident

Re: Show HN: An API that takes a URL and returns a file with browser screenshots

#96

Post by anonzzzies »

jchw wrote: post #70

One thing to be cognizant of: if you're planning to run this sort of thing against potentially untrusted URLs, the browser might be able to make requests to internal hosts in whatever network it is on. It would be wise, on Linux, to use network namespaces, and block any local IP range in the namespace, or use a network namespace to limit the browser to a wireguard VPN tunnel to some other network.

Is there a self hosted version that does this properly?

Re: Show HN: An API that takes a URL and returns a file with browser screenshots

#97

Post by sixothree »

xg15 wrote: post #51
cmgriffing wrote: post #33

Earlier quoted context omitted.

Quick note: when trying to do full page screenshots, Chrome does a screenshot of the current view, then scrolls and does another screenshot. This can cause some interesting artifacts when rendering pages with scroll behaviors. Firefox does a proper full page screenshot and even allows you to set a higher DPS value. I use this a lot when making video content. Check out some of the args in FF using: `:screenshot --help…

I mean, if you have some of those annoying "hijack scrolling and turn the page into some sort of interactive animation experience" sites, I don't think "full page" would even be well-defined.

Pretty sure this refers to sticky headers. They have caused me many headaches when trying to get a decent screenshot.

Re: Show HN: An API that takes a URL and returns a file with browser screenshots

#98

Post by yjftsjthsd-h »

blueflow wrote: post #92
yjftsjthsd-h wrote: post #79

Earlier quoted context omitted.

> now try and go ahead how you could isolate these instances so they cannot see each other. this leads into a rabbit hole of bad design. Okay, done: PROFILEDIR="$(mktemp -d)" firefox --no-remote --profile "$PROFILEDIR" --screenshot $PWD/output.png https://xkcd.com rm -r "$PROFILEDIR" What's the rabbit hole?

Whats with the dbus interface?

What?

(If you're trying to point out that two firefoxes are capable of talking to each other via system IPC, then yes, fully isolating any two programs on the same machine requires at least containers but probably full VMs, which has nothing to do with Firefox itself, and you'd need to explain why in this situation we should care)

Re: Show HN: An API that takes a URL and returns a file with browser screenshots

#99

Post by theogravity »

kevinsundar wrote: post #57
theogravity wrote: post #55

Earlier quoted context omitted.

there's also our product, Airtop ( https://www.airtop.ai/ ), which is under the scraping specialist / browser automation category that can generate screenshots too.

Hey I'm curious what your thoughts are on whether you need a full blown agent that moves the mouse and clicks to extract contents from webpages or a more simplistic tool that can just scrape pages + take screenshots and pass it through an LLM is generally pretty effective? I can see niches cases likes videos or animations being better understood by an agent though.

Airtop is designed to be flexible, you can use it as part of a full-blown agent that interacts with webpages or as a standalone tool for scraping and screenshots.

One of the key challenges in scraping is dealing with anti-bot measures, CAPTCHAs, and dynamic content loading. Airtop abstracts much of this complexity while keeping it accessible through an API. If you're primarily looking for structured data extraction, passing pages through an LLM can work well, but for interactive workflows (e.g., authentication, multi-step navigation), an agent-based approach might be better. It really depends on the use case.

Re: Show HN: An API that takes a URL and returns a file with browser screenshots

#100

Post by ALittleLight »

jot wrote: post #85
ALittleLight wrote: post #61

Earlier quoted context omitted.

Maybe I'm misreading. It sounds like you're stepping the user up a pricing tier - e.g. going from 50 a month to 100 and then charging at the better rate. I would also worry about a bug on my end that fires off lots of screenshots. I would expect a quota or limit to protect me from that.

That’s right. On our standard self-service plans we automatically charge a better rate as volume increases. You only pay the difference between tiers as you move through them. It’s rare that anyone makes that kind of mistake. It probably helps that our rate limits are relatively low compared to other APIs and we email you when you get close to stepping up a tier. If you did make such a mistake we would, like all good…

I'm not a customer, so don't take what I say too seriously, but to me it seems like you are unilaterally making a purchasing decision on my behalf. That is, I agreed to pay you 50 dollars a month and you are deciding I should pay 100 (or more) - to "upgrade" my service. My intuition is that this is probably not legal, and, if I were a customer, I would not pay for a charge that I didn't explicitly agree to - if you tried to charge me I would reject it at the credit card level.

If I sign up for a service to pay X and get Y, then I expect to pay X and get Y - even if my automated tools request more than Y - they should be rejected with a failure message (e.g. "quota limit exceeded").

Post reply on HN