Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Kage – Shadow any website to a single binary for offline viewing

github.com

Post reply on HN
Print view View original

91–100 of 152 posts

Re: Show HN: Kage – Shadow any website to a single binary for offline viewing

#91

Post by embedding-shape »

vqtska wrote: post #72
simonw wrote: post #38

I was intrigued to see how the demo GIF in the README was generated: https://github.com/tamnd/kage/blob/01e75b87ecc893bbba7943c63... Turns out it's using another project by the same author: https://github.com/tamnd/ascii-gif The script used for the demo is at https://github.com/tamnd/kage/blob/01e75b87ecc893bbba7943c63... and has a comment showing how to run it: ascii-gif render docs/demo/kage.tape -o docs/static/dem…

You can also do an animated svg which is way smaller than a gif because it's just text keyframes ( https://github.com/vytskalt/pseudoc/blob/main/assets/factori... )

Very cool, never thought of that! "way smaller" is almost an understatement, when it's 50kb :P Neat that it loads in GitHub READMEs as well, which is probably a large reason people use .gif today.

Re: Show HN: Kage – Shadow any website to a single binary for offline viewing

#92

Post by nikisweeting »

tamnd wrote: post #81
nikisweeting wrote: post #73

Earlier quoted context omitted.

--no-sandbox is needed in docker, maybe they assume it will mostly run in docker?

Exactly. For downloading, Kage requires Chrome or Chromium. Running it inside Docker makes setup easier and keeps cleanup simple: https://github.com/tamnd/kage/blob/main/Dockerfile Btw, let me think the way to only enable this when running inside Docker.

Docker is designed to be undetectable by default, the best way I have found is to set env IN_DOCKER=True manually in your Dockerfile + check that there is no $DISPLAY configured + that you're on linux. Usually if all/most of those are true you can safely add --no-sandbox --disable-setuid-sandbox --disable-dev-shm-usage etc. all the docker-specific flags. Thats what we do in https://github.com/ArchiveBox/ArchiveBox/blob/dev/Dockerfile...

Re: Show HN: Kage – Shadow any website to a single binary for offline viewing

#93

Post by nikisweeting »

tamnd wrote: post #3
maxloh wrote: post #2

I find SingleFile [0] to be a much more robust version of this. It strips out all the JavaScript too, but also packs everything into a single HTML file that is easy to transfer. Binary assets (like web fonts and images) are packed as base64 strings. They also offer a CLI powered by Puppeteer. [1] [0]: https://github.com/gildas-lormeau/singlefile [1]: https://github.com/gildas-lormeau/single-file-cli

It seems this repo only saves one web page? What I'm implementing here is mirroring a whole website, with all its subpages, so you can browse it all offline. For example, all essays from paulgraham.com.

Singlefile supports scoped recursive crawls too: https://github.com/gildas-lormeau/single-file-cli#:~:text=an...

I highly recommend reading the singlefile source or https://archiveweb.page/ to see how they handle closed shadow DOMs, cross-origin iframes, websockets, media urls, deduping large assets, etc.

Re: Show HN: Kage – Shadow any website to a single binary for offline viewing

#94

Post by godot »

the readme uses paulgraham.com as an example (which is text articles mostly) and I never use "Save As" for a web page (for the reasons the author states), I always just print as PDF and save the PDF file.

for an entire website though of many pages I can see this can be useful.

Re: Show HN: Kage – Shadow any website to a single binary for offline viewing

#96

Post by gwern »

wolttam wrote: post #10

One use I'd have for this is company wikis that you want to give folks easy offline access to (maybe the wiki has documentation that's useful at sites that don't have cellular coverage). Cool! It would be especially cool to have a version that didn't require the separate serving process - even though it's nifty you can package up a whole site as a single binary. Maybe a single HTML entrypoint shim with a bit of javas…

> Maybe a single HTML entrypoint shim with a bit of javascript that could index into an archive (potentially embedded) of the site's content?

So something like SingleFileZ https://github.com/gildas-lormeau/SingleFileZ or Gwtar https://gwern.net/gwtar ?

Re: Show HN: Kage – Shadow any website to a single binary for offline viewing

#97

Post by smeej »

tamnd wrote: post #14
wolttam wrote: post #10

One use I'd have for this is company wikis that you want to give folks easy offline access to (maybe the wiki has documentation that's useful at sites that don't have cellular coverage). Cool! It would be especially cool to have a version that didn't require the separate serving process - even though it's nifty you can package up a whole site as a single binary. Maybe a single HTML entrypoint shim with a bit of javas…

Submitting this to Hacker News is the right place! Thanks for your idea. I will consider implementing that :) Also, in my mind, I already have a script/program to convert HTML to Markdown, so it could actually store everything on disk as a folder of Markdown files, and then commit them to a Git repo.

I would use the shit out of this. I'm a heavy user of Logseq (OG, the md file-based version). Would LOVE to save my favorite web resources this way.

Re: Show HN: Kage – Shadow any website to a single binary for offline viewing

#99

Post by recursive »

danielheath wrote: post #59
recursive wrote: post #50

Earlier quoted context omitted.

I am quite familiar with this and it is factually false

Js modules don’t work on file urls (classic js does).

They can be made to work with blob urls. I have done this.
Post reply on HN