This looks pretty cool, I'm excited to try it. The choice of lesser known format vs WARC is interesting, afaik WARC can do pretty complex stuff (like replaying youtube videos).
Show HN: Self-hosted offline Internet from your browsing history
121–130 of 166 posts
Re: Show HN: Self-hosted offline Internet from your browsing history
#122Anyone know of something like this that can sit on a network, possibly as a web proxy?
Webrecorder.io is the one you want if you're looking for a proxy archiver. PYWB (the underlying library) is by far the best proxy archiver around imo.
Now called https://github.com/Rhizome-Conifer/conifer ^
Re: Show HN: Self-hosted offline Internet from your browsing history
#123Earlier quoted context omitted.
Ah, that brings back memories. Didn't Palm OS have something similar? I think it was Plucker [1], but I'm not too sure. [1]
Yep. With Plucker, I could download the New York Times web site (I think via RSS) before I went to work, sync it to my Palm Pilot, and then read it on my lunch.
Many, many years ago, I used a custom build of Plucker to build an offline reader for the schedule of a large conference in the UK - to the point of rebuilding the schduule file daily to include updates, and using an old laptop with an irda port running pilot-link in a loop to sync the new schedule to anyone who wanted it... That was fun.
Re: Show HN: Self-hosted offline Internet from your browsing history
#124This looks pretty cool, I'm excited to try it. The choice of lesser known format vs WARC is interesting, afaik WARC can do pretty complex stuff (like replaying youtube videos). See: https://github.com/Rhizome-Conifer/conifer
Coming from the author of ArchiveBox, that's greatly respected to see your comment here, thank you!
Re: Show HN: Self-hosted offline Internet from your browsing history
#125This uses Chrome DevTools Protocol in a pretty clever way. I used it to archive a highly interactive website and it worked like a charm. The README states: "It runs connected to a browser, and so is able to access the full-scope of resources (with, currently, the exception of video, audio and websockets, for now)" I wonder what kind of limitations makes it hard to intercept those resources like the rest of the conten…
Video and audio is probably just a matter of not having gotten around to it yet. WebSockets are another matter. I’m not sure what one would do with a two-way channel in a general sense. It’s often not an idempotent operation.
This guy is right I just haven't gotten around to the audio and video it's a little bit more work I think. websockets is not a normal request response thing that I think is easy to cache so it's a little bit more complex and I haven't really begun to think about it in a way that I feel will be a good approach... Or any approach that I think could work.
Re: Show HN: Self-hosted offline Internet from your browsing history
#126This uses Chrome DevTools Protocol in a pretty clever way. I used it to archive a highly interactive website and it worked like a charm. The README states: "It runs connected to a browser, and so is able to access the full-scope of resources (with, currently, the exception of video, audio and websockets, for now)" I wonder what kind of limitations makes it hard to intercept those resources like the rest of the conten…
Video files are massive, so it may just be the case that archiving videos takes so long they didn't want to support it.
That's also a consideration. And I haven't thought of a way to approach it where we get the nice sort of size of the archives that we have currently, and when adding video it would just balloon and I think that could be surprising so I haven't thought of a good approach to gel all that.
Re: Show HN: Self-hosted offline Internet from your browsing history
#127Given the hard 'no' in the FAQ, does anyone know about a similar project for Firefox?
Look now that Firefox seems to be supporting the webdriver protocol more closely, the progress of FF to which I don't follow that closely, so it might have caught up quite a lot, but it will be possible it'll just be a bit more work and I don't care to do that without knowing you know how many people it would really affect. I feel quite sorry for the people to who don't enjoy seeing they can't use Ff.
Also, I don't want to do it if I have to change the way I use the protocol so I would want the same methods to be pretty much available and to work the same way.
Re: Show HN: Self-hosted offline Internet from your browsing history
#128Super cool! I build a similar project last year but I haven't put much work into it because I had built that version as an alternative browser and there's way too much overhead for too little gain. I like the idea of using Dev Tools to get the requests. If I revisit my project, I would build it to register itself as the system proxy, including SSL MITM. For future work on this project, consider a search engine built…
Yes i also want to add the search. Just not sure the best full text search using Node for filesystem files
Re: Show HN: Self-hosted offline Internet from your browsing history
#129I miss RSS primarily because I was able to search for stuff either I've read or I care about... I am too embarrassed to admit that a disproportionate amount of my time are spent on looking for a sentence or god forbid a tweet I vaguely remember reading last week. SO yes, consider this a vote for that sexy full text search please.
All right thanks I will!