Show HN: CLI tool for saving web pages as a single file • Hacker News Board

Re: Show HN: CLI tool for saving web pages as a single file

#141

Post by gildas » Fri, Aug 23, 2019, 1:48 PM UTC

mxuribe wrote: post #140

fit2rule wrote: post #86

I've been printing to PDF for decades now, and nothing comes close to the ease of use and versatility of 2 decades worth of interesting web pages .. I have pretty much every interesting article, including many from HN, from decades of this habit. Need to find all articles relating to 'widget'? $ ls -l ~/PDFArchive/ | grep -i widget This has proven so valuable, time and again .. there is a great joy in not having to m…

Its funny, on a rare occasion, I too have saved some content as a PDF - more so for archiving rather than for offline viewing...But i guess i never thought to scale it for all/most of my bookmarks. It seems so obvious now after reading your comment. However, my experience with PDFs has been negative. From filesize to slow booting of myriad pdf viewers, etc., it just seems like viewing stuff in native html, text is be…

I think SingleFile or SingleFileZ [1] would indeed solve your needs.

[1] https://github.com/gildas-lormeau/SingleFileZ

Re: Show HN: CLI tool for saving web pages as a single file

#142

Post by pgcj_poster » Fri, Aug 23, 2019, 1:49 PM UTC

mikekchar wrote: post #134

flatroze wrote: post #90

Earlier quoted context omitted.

Which license would you recommend to release this software under to reach the broad adoption yet permissive terms, if not the Unlicense?

I honestly don't know. That's why the question :-) Is CC0 good for software? It seems to be a bit more complete from a non-US view point, but I don't know if there are lurking situations. Possibly MIT is better -- it's pretty darn permissive. I'm really just soliciting opinions.

Yes, CC0 is the only Creative Commons License suitable for software. It's endorsed by the Free Software Foundation [1], although not the Open Source Initiative. I use it for everything that I don't want to copyleft.

[1] https://www.gnu.org/licenses/license-list.html#PublicDomain

Re: Show HN: CLI tool for saving web pages as a single file

#143

Post by mxuribe » Fri, Aug 23, 2019, 1:52 PM UTC

gildas wrote: post #141

mxuribe wrote: post #140

Earlier quoted context omitted.

Its funny, on a rare occasion, I too have saved some content as a PDF - more so for archiving rather than for offline viewing...But i guess i never thought to scale it for all/most of my bookmarks. It seems so obvious now after reading your comment. However, my experience with PDFs has been negative. From filesize to slow booting of myriad pdf viewers, etc., it just seems like viewing stuff in native html, text is be…

I think SingleFile or SingleFileZ [1] would indeed solve your needs. [1] https://github.com/gildas-lormeau/SingleFileZ

Yep, I'll look into this, thanks!

Re: Show HN: CLI tool for saving web pages as a single file

#144

Post by [deleted] » Fri, Aug 23, 2019, 2:39 PM UTC

mikaelmorvan wrote: post #128

lucideer wrote: post #122

Earlier quoted context omitted.

The capture includes JS, so this should work for most JS-dependent sites, with the exception of scripts loading other additional assets. Tbh, often those are superfluous, or egregious examples of bad web dev, so it seems a reasonable solution for most cases. SingleFile is a different approach, but it's a lot more involved/less convenient than a cli, and loading in something like WebDriver on the cli for this would be…

superfluous, or egregious examples of bad web dev?? Do you know what Web 2.0 is? Do you know what are React, Angular, and the other JS Frameworks? When you create a modern webapp, a lot of data are retrieved from servers as Json and formated in the browser in Javascript. Even sometimes Css is generated on browser-side. Even more, on webapp where user login is taken into account, the display is modified accordingly. T…

[deleted]

Re: Show HN: CLI tool for saving web pages as a single file

#145

Post by psychrometer » Fri, Aug 23, 2019, 2:43 PM UTC

interfixus wrote: post #109

laumars wrote: post #72

Earlier quoted context omitted.

AUR are packages that aren't in Arch's repo system. Granted tools like yaourt do make installing AUR packages nearly as easy as pacman but anyone can upload anything to AUR thus you are expected to vet the packages yourself (hence why tools like yaourt repeatedly prompt you to read the build scripts et al before running them).

> AUR are packages that aren't in Arch's repo system AUR, Arch User Repository . > thus you are expected to vet the packages yourself Obviously, as a long time Arch user I didn't know this.

You emphasized the wrong word.

It is the Arch User Repository.

> DISCLAIMER: AUR packages are user produced content. Any use of the provided files is at your own risk.[0]

[0] https://aur.archlinux.org

Re: Show HN: CLI tool for saving web pages as a single file

#146

Post by forgotmypw » Fri, Aug 23, 2019, 2:49 PM UTC

OJFord wrote: post #129

forgotmypw wrote: post #110

Earlier quoted context omitted.

I do all of my development in a VM, which allows me to take snapshots and have a portable GNU workstation that's decoupled from my desktop and hardware. Meanwhile, my desktop remains clean and ready to play media in native environment with good hardware support. I used to think it'd be slow, until I tried it. My computer is 8+ years old, and it works fine. I mostly do text work.

I used to use a VM (motivation being to run Linux on my locked-down corporate-imaged Macbook) and it was usable, but not as fast it could be if it had free reign over all CPU/RAM. But, at least the way I was doing it, it's not adding any security as discussed here, since you're doing everything in the VM so anything in the VM has access to everything just as if everything on the host anyway.

Scripts I run have access to my development environment for a free software project I already keep in a public repo.

What they don't have access to is my actual desktop, nor yesterday's snapshot of the VM desktop.

It's obviously not as fast as running GNU on bare metal, but it's fast enough for text work.

Re: Show HN: CLI tool for saving web pages as a single file

#147

Post by roblabla » Fri, Aug 23, 2019, 3:03 PM UTC

OJFord wrote: post #102

FreeHugs wrote: post #61

One thing I always wonder when I see native software posted here: How do you guys handle the security aspect of executing stuff like this on your machines? Skimming the repo it has about a thousand lines of code and a bunch of dependencies with hundreds of sub-dependencies. Do you read all that code and evaluate the reputation of all dependencies? Do you execute it in a sandboxed environment? Do you just hope for the…

Some people run ~everything in individual containers, with original commands being aliases for the docker (or whatever) equivalent. I've shunned that assuming it'd be a big slow down, but I do keep meaning to at least try it, uh, after I knock it. (No, containers like anything else aren't and haven't been completely secure all of the time since and forever, but it'd take a more sophisticated - and certainly deliberat…

At least with containers, if something manages to do damage, it will have to do it through a security vulnerability - which will eventually get patched if caught. The current state of affairs is that malware can do damage without even needing to use a security vulnerability whatsoever. It's "working as designed", and will keep working forever.

Re: Show HN: CLI tool for saving web pages as a single file

#148

Post by roblabla » Fri, Aug 23, 2019, 3:07 PM UTC

masklinn wrote: post #106

Huggernaut wrote: post #85

Earlier quoted context omitted.

I think that's an extremely uncharitable view on containers. There has been a massive amount of work put into securing containers for a variety of use cases using both layers available and by adding to the kernel.

> I think that's an extremely uncharitable view on containers. It's an objective one. > There has been a massive amount of work put into securing containers for a variety of use cases using both layers available and by adding to the kernel. That doesn't change the fact that security was never the primary goal for containers, so secure containers were and are a bunch of tricks, kludges and prayers being built up in th…

Can you please quote how containers are not built with security in mind? What would even be the point of user namespacing, network namespaces, filesystem namespaces, etc... if not security?

Re: Show HN: CLI tool for saving web pages as a single file

#149

Post by ur-whale » Fri, Aug 23, 2019, 3:09 PM UTC

Does not compile with some byzantine message about let in const funcs being unstable.

Re: Show HN: CLI tool for saving web pages as a single file

#150

Post by gambler » Fri, Aug 23, 2019, 3:18 PM UTC

FreeHugs wrote: post #61

One thing I always wonder when I see native software posted here: How do you guys handle the security aspect of executing stuff like this on your machines? Skimming the repo it has about a thousand lines of code and a bunch of dependencies with hundreds of sub-dependencies. Do you read all that code and evaluate the reputation of all dependencies? Do you execute it in a sandboxed environment? Do you just hope for the…

This is a good question. I think you can make it even better by generalizing the problem. How on earth do developers hope to advance general computing forward when simply running programs isn't a solved problem? Most software engineers I know don't run docker on their home PCs. What about people who aren't in IT? Does anyone here even care? The general attitude I see is "plebs don't need to run anything they can't get outside of an app store". It's a horrible attitude.

I very much like this quote from Alan Kay:

"It doesn't matter what the computer can do, if it can't be learned by billions of people."

There is no good technical reason why modern operating systems can't work out some some scheme for sanboxing arbitrary programs by default. It is obviously necessary. I imagine something like "applications" folder where every subfolder automatically becomes an isolated "container". It would have to be designed with security as the primary concern, though; unlike current container solutions.