31–40 of 216 posts
I suspect for saving videos, a good approach would be some sort of proxy + headless browser combination, where the proxy is responsible for saving a copy of all data the browser requests for.
Thoughts?
Earlier quoted context omitted.
Always struck me as quite odd MHTML fell out of favor. Back in the day when I wanted to preserve a web page it was the logical choice since you could just click "save as archive".
Blame XMLHttpRequest, Flash, JS, and embedded video. It doesn't make sense to archive a document when the necessary interactive content elements will essentially fail when opened offline.
How does this work on neverending webpages/forever scroll? How will it behave if you need to authenticate before browsing the page?
This would be a perfect fit for IPFS. I love the idea of having just one file in a permanent link.
This could also be an interesting alternative to PDF, especially with web fonts embedded as data URLs.
Imagine a paper in the form of a single HTML file, which has (a subset of) the data included, the graphs zoomable, the colors chanegable (to whatever vision problems you have) - maybe even the algorithm to play around with!
Jupyter Notebooks already go in that way. only without the single-file, open in browser aspect, I think.
Ah, I've been thinking about making something like this. You beat me to it. I've been using the SingleFile add-on until now. I'll definitely give this a try.
Ahh, to me it looks like it creates an amalgamation of the web page+contents. How does this work on neverending webpages/forever scroll? How will it behave if you need to authenticate before browsing the page?
It seems to work for basic pages quite well, I think that lazy load will work for most pages as long as the JavaScript is embedded (no -j flag provided) and the Internet connection is on. It saves what's there when the page is loaded, the rest is a gamble since every website implements infinite scroll differently.
Authentication is another tricky part -- it's different for every browser. I will try to convert it into a web extension of sorts, so that pages could be saved directly from the browser while the user is authenticated.
MHTML is pretty good for this already btw (not to take away from this neat project though :)). Similarly stores assets as base64'd data URIs and saves it as a single file. Can be enabled in Blink-based browsers using a settings flag and previously in Firefox using addons (also in the past natively in Opera and IE).
Does it? IIRC it stores assets as MIME attachments, hence the "M": the result is not HTML (which this would I assume be), it's a multipart MIME message whose root is an HTML document.
edit: in fact when downloading mht files osx / safari misrecognises them as exported emails and appends the "eml" extension.
I'm glad there's more people taking a look at the use case, and I'd be interested to see a list of similar solutions.
If you combine this with Chrome's headless mode, you can prerender many pages that use JavaScript to perform the initial render, and then once you're done send it to one of these tools that inlines all the resources as data URLs.
/Applications/Google\ Chrome.app/Contents/MacOS/Google\ Chrome ./site/index.md.html --headless --dump-dom --virtual-time-budget=400