Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Self-hosted offline Internet from your browsing history

github.com

Post reply on HN
Print view View original

111–120 of 166 posts

Re: Show HN: Self-hosted offline Internet from your browsing history

#111

Post by CGamesPlay »

Super cool! I build a similar project last year but I haven't put much work into it because I had built that version as an alternative browser and there's way too much overhead for too little gain. I like the idea of using Dev Tools to get the requests. If I revisit my project, I would build it to register itself as the system proxy, including SSL MITM.

For future work on this project, consider a search engine built on top of the downloaded files! Also, gzip your JSON.

My older project for anyone interested: https://github.com/CGamesPlay/chronicler

Re: Show HN: Self-hosted offline Internet from your browsing history

#112

Post by gojomo »

walski wrote: post #52
toomuchtodo wrote: post #44

Earlier quoted context omitted.

Why a zip file instead of a WARC file? https://en.wikipedia.org/wiki/Web_ARChive

see: https://github.com/c9fe/22120#why-not-warc-or-another-format... > Both WARC and MHTML require mutilatious modifications of the resources so that the resources can be "forced to fit" the format. At 22120, we believe this is not required

This is false. If you're doing WARC correctly, HTTP resources/responses are stored verbatim.

Perhaps the only possible purer format would be packet captures, say of the full HTTPS session, along with the session keymatter and connection metadata to later extract the verbatim HTTP resources. That'd be interesting, but I doubt that's what this "22120 format" (for which I see no documentation links) does.

Re: Show HN: Self-hosted offline Internet from your browsing history

#113

Post by blackflame7000 »

peterburkimsher wrote: post #80

It looks like something I'd appreciate! I make a significant effort to archive things that I think I'll need. Unfortunately it didn't work when I just tried installing it now (macOS 10.13.6, node v14.8.0). MacBook-Pro:Desktop peter$ npx archivist1 npx: installed 79 in 8.282s Preferences file does not exist. Creating one... Args usage: Updating base path from undefined to /Users/peter... Archive directory (/Users/pete…

Here is your error: connect ECONNREFUSED 127.0.0.1:9222

Re: Show HN: Self-hosted offline Internet from your browsing history

#114

Post by ernsheong »

I built PageDash (https://pagedash.com), which performs more as a web clipper. Try something like this if you want something more lightweight and hassle-free. It does put the pages into the cloud. PageDash has been in operation since 2017 and is more or less self-sustaining (though not enough to make me quit my job).

Previous Show HN: https://news.ycombinator.com/item?id=15653206

Re: Show HN: Self-hosted offline Internet from your browsing history

#115

Post by slumdev »

derefr wrote: post #61
Donald wrote: post #56

Earlier quoted context omitted.

You must be young :) These browsers were born in the era of dialup Internet that had per minute charges and/or long distance charges. At the very least you were tying up your family's phone line. Basically it's like paying for every minute your cable modem is plugged in. For the feature itself: Netscape had integration with the modem connectivity for the OS and would initiate a connection when you tried to visit a re…

I ran a BBS, my friend :) I'm quite familiar with modems. I just never used Windows (or the web!) until well past the Netscape era, so I'm not too familiar with the intersection of modems and early web browsers. > Netscape had integration with the modem connectivity for the OS and would initiate a connection when you tried to visit a remote page. That's not "integration with modem connectivity", that's just going thr…

> In which case, would the page, in fact, be served from the client-side cache, or would you just get nothing?

IIRC, some of the browsers, in "offline mode", would actually serve pages from the local cache, not even attempting to fetch them from the remote server. If you attempted to navigate to a page that hadn't been cached, you got an error of some kind.

Re: Show HN: Self-hosted offline Internet from your browsing history

#116

Post by girishso »

abnry wrote: post #73

I am coding my own hacked together bookmarks manager. I can save any page with a click of the button using SingleFile (a fantastic Chrome extension, by the way!). Then a cronjob runs and puts it into a folder to be processed into a database, which generates a static html index and puts it in my Google Drive. Then it syncs offline on my chromebook. Which means that without internet, I can put my chromebook in tablet m…

Been there done that. Real issue is ranking the search results by relevance to the search query.

Re: Show HN: Self-hosted offline Internet from your browsing history

#118

Post by SirLotsaLocks »

avmich wrote: post #69

Awesome thing, but - > Can I use this with a browser that's not Chrome-based? > No. Note that a (rather similar) thing I've participated in in 2002 was browser-neutral.

yeah I was really excited until I got to that point.

Re: Show HN: Self-hosted offline Internet from your browsing history

#119

Post by shakna »

rzzzt wrote: post #70
derefr wrote: post #50

Earlier quoted context omitted.

I'm not sure that was the use-case. With pre-DHTML HTML4, there really just wasn't anything on a page that could continue to interact with the server after the page finished loading. So, presuming the button was for your described use-case, what would the difference be between "going offline" and just... not clicking any more links? (It's not like Netscape could or should signal your modem to hang up — Netscape doesn…

Server-Sent Events? How old is that mechanism?

2006 was when Opera added the functionality which would become Server Sent. [0]

[0] https://dev.opera.com/blog/event-streaming-to-web-browsers/

Re: Show HN: Self-hosted offline Internet from your browsing history

#120

Post by shakna »

derefr wrote: post #61
Donald wrote: post #56

Earlier quoted context omitted.

You must be young :) These browsers were born in the era of dialup Internet that had per minute charges and/or long distance charges. At the very least you were tying up your family's phone line. Basically it's like paying for every minute your cable modem is plugged in. For the feature itself: Netscape had integration with the modem connectivity for the OS and would initiate a connection when you tried to visit a re…

I ran a BBS, my friend :) I'm quite familiar with modems. I just never used Windows (or the web!) until well past the Netscape era, so I'm not too familiar with the intersection of modems and early web browsers. > Netscape had integration with the modem connectivity for the OS and would initiate a connection when you tried to visit a remote page. That's not "integration with modem connectivity", that's just going thr…

> Was HTTP client-side caching even a thing in the early 90s? Did disks have the room to hold client-side caches?

I was considered mad at the time for upping my cache to a whopping 2Mb by friends, but Netscape's cache was highly configurable.

Things like cache pages, but not images, always cache bookmarked pages, cache iframe pages (which were often navigation), etc. Netscape 4 added CSS to that mix.

Post reply on HN