Live data from Hacker News

Show HN: Pocket Stream Archive – A personal Way-Back Machine

github.com

51–60 of 70 posts

Re: Show HN: Pocket Stream Archive – A personal Way-Back Machine

#52

Earlier quoted context omitted.

PDFs are really not suitable for archiving websites since they're designed around pages and the web does not have pages. A better option is to render a page with JS turned on and save the resulting HTML.

I was doing this before with the chrome --dump-dom flag but the output I was getting was garbage, and no more useful than the simpler wget download. PDF turned out to produce really nice, readable archives about 75% of the time, so I kept it in. Text-based sites tend to do a good job of having PDF-friendly styling.

I googled it, and --dump-dom simply dumps the output of document.body.innerHTML. I have not used headless chrome at all, but I imagine it would not be that hard to get it to dump document.documentElement.outerHTML instead.

(execute JavaScript on the page is most likely possible?)

Re: Show HN: Pocket Stream Archive – A personal Way-Back Machine

#53
post #19

Screenshotting or PDFing of a website is an increasingly important archiving tool, to supplement wget. I've come across a lot of websites that won't render any content if not connected to a live server.

I couldn't agree more. I wish more sites would load without needing multiple seconds of JS execution and AJAX. One of my TODOs is to get full-page screenshots working as well.

> One of my TODOs is to get full-page screenshots working as well.

https://medium.com/@dschnr/using-headless-chrome-as-an-autom...

Re: Show HN: Pocket Stream Archive – A personal Way-Back Machine

#55

If it would be connected to Pinboard.in as an alternative to Pocket I would be screaming with joy. :-)

If you can get me a sample pinboard export to look at, I'll whip up a regex that makes it work.

Oh, thanks!

There's three backup formats: XML (same as Delicious v1 API), HTML (legacy Netscape format almost everyone can read) and JSON.

I can get back with an XML file later. There's also an API that might be of interest.

Here's someone who blogged about that:

http://behindcompanies.com/2011/12/a-guide-to-backing-up-pin...

The question was asked if you can use your API endpoint URL (https://[pinboardusername]:[pinboardpassword]@api.pinboard.i...) straight into ifttt. Maciej confirmed that it should work, the problem is that you’re essentially storing your login credentials in a 3rd party service, and you don’t know if they’re storing and transmitting it securely.

Re: Show HN: Pocket Stream Archive – A personal Way-Back Machine

#56

Use zotero and you have your own personal Pocket with snapshots. In addition, you can add tags, organize stuff into folders, etc. https://www.zotero.org/

Zotero is awesome! It doesn't provide a publishable stream of recently added articles though afaik.

Re: Show HN: Pocket Stream Archive – A personal Way-Back Machine

#57
post #48

I still use Firefox ScrapBook for this: https://addons.mozilla.org/en-US/firefox/addon/scrapbook/ Just for articles, mind you, not entire websites.

Ditto. I just hope the transition to web extension will be smooth or at least happening at all.

Re: Show HN: Pocket Stream Archive – A personal Way-Back Machine

#58
post #38

You see something is flawed in Redux at the point you have to pass strings (uppercase constants defined somewhere) around, import them in every file, pass them as identifiers of what you should do with each data. Strings!

Did you comment on the wrong article by accident? https://news.ycombinator.com/item?id=14273549

Yes!

Thank you.

Re: Show HN: Pocket Stream Archive – A personal Way-Back Machine

#59
post #19

Screenshotting or PDFing of a website is an increasingly important archiving tool, to supplement wget. I've come across a lot of websites that won't render any content if not connected to a live server.

I couldn't agree more. I wish more sites would load without needing multiple seconds of JS execution and AJAX. One of my TODOs is to get full-page screenshots working as well.

You might find this useful if you use Firefox:

- shift+f2

- screenshot urcodiaz.png --fullpage

Re: Show HN: Pocket Stream Archive – A personal Way-Back Machine

#60

Earlier quoted context omitted.

I couldn't agree more. I wish more sites would load without needing multiple seconds of JS execution and AJAX. One of my TODOs is to get full-page screenshots working as well.

You might find this useful if you use Firefox: - shift+f2 - screenshot urcodiaz.png --fullpage

Thank you for this tip! I have always installed Abduction plugin if I needed this, but the less plugins the better.
Post reply on HN