Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

ArchiveBox/ArchiveBox: open-source self-hosted web archiving

github.com

Post reply on HN
Print view View original

21–27 of 27 posts

Re: ArchiveBox/ArchiveBox: open-source self-hosted web archiving

#21

Post by sleavey »

soneil wrote: post #17
sleavey wrote: post #13

Earlier quoted context omitted.

> I use it like permanent bookmarks. I create browser bookmarks regularly but I wouldn't be bothered to SSH into my server to also tell it to grab a copy of the URL. Automating this with a browser plugin would be cool.

There is a web UI, so I have a JS "bookmarklet" in my toolbar. If you go to the web UI and Add a url, right at the bottom of the page there's a link you can bookmark to do the same. So that's my workflow - click Archive (or cmd-alt-1), then hit enter. Done.

Aha, that's great! I might give this a go this weekend then.

Re: ArchiveBox/ArchiveBox: open-source self-hosted web archiving

#22

Post by thedanbob »

rambambram wrote: post #12
thedanbob wrote: post #8

Earlier quoted context omitted.

I’ve used it to save a lot of pages related to ham radio. I have several 30-40 year old radios and I’m afraid one day information about them will just drop off the internet.

Cool! How is the saving experience? Is it quick? And looking up? Or is it more just 'saving for later use'?

Fairly quick, depending of course on the weight of the page(s) and how many archive methods you enable (I have several turned off because they are redundant for my purposes). The "adding to archive" interstitial still tends to time out which is a little annoying, but the actual archive process is backgrounded so it doesn't matter.

The recommended install includes a search engine that works well, aside from a few false positives due to being fuzzy search. I don't have much in it yet so I can't say what the performance is like once you reach e.g. thousands of pages, but I imagine it would still perform well except maybe for mass operations like rebuilding the entire index.

Re: ArchiveBox/ArchiveBox: open-source self-hosted web archiving

#23

Post by nikisweeting »

sleavey wrote: post #21
soneil wrote: post #17

Earlier quoted context omitted.

There is a web UI, so I have a JS "bookmarklet" in my toolbar. If you go to the web UI and Add a url, right at the bottom of the page there's a link you can bookmark to do the same. So that's my workflow - click Archive (or cmd-alt-1), then hit enter. Done.

Aha, that's great! I might give this a go this weekend then.

There's also a real browser extension in the works by one of our users: https://github.com/ArchiveBox/ArchiveBox/issues/577#issuecom...

Re: ArchiveBox/ArchiveBox: open-source self-hosted web archiving

#24

Post by nikisweeting »

Thanks for posting this @rcarmo!

One neat tiny implementation detail of ArchiveBox that I just highlighted on HN today is our use of asymptotic progress bars when we don't know how long archiving a page is going to take: https://news.ycombinator.com/item?id=27860022

Re: ArchiveBox/ArchiveBox: open-source self-hosted web archiving

#25

Post by ammar_x »

When I tried to add my bookmarks from a file, one of the website seems to not be online anymore; archivebox just stops execution when it encounters this issue.

How can I make archivebox ignore such errors and continue with the rest of the websites?

Command used:

  archivebox add

Re: ArchiveBox/ArchiveBox: open-source self-hosted web archiving

#26

Post by epilys »

thedanbob wrote: post #8
rambambram wrote: post #5

I really need to try this out soon. I keep bumping into it online. What do people on HN use it for? What I do right now "to collect, save, and view sites you want to preserve offline" is by use of a Firefox plugin called WebScrapBook. Click-click-done, and I have a local searchable (!) copy of a webpage exactly as it looked in the browser. With styles and all, in one file. WebScrapBook is pretty highly configurable.…

I’ve used it to save a lot of pages related to ham radio. I have several 30-40 year old radios and I’m afraid one day information about them will just drop off the internet.

Consider having archive.org save the websites, in case they are not already.

Re: ArchiveBox/ArchiveBox: open-source self-hosted web archiving

#27

Post by nikisweeting »

ammar_x wrote: post #25

When I tried to add my bookmarks from a file, one of the website seems to not be online anymore; archivebox just stops execution when it encounters this issue. How can I make archivebox ignore such errors and continue with the rest of the websites? Command used: archivebox add

It should not stop execution, can you open an issue with your log output or a screenshot.

Also note you got the arguments backwards, make sure to put the file redirect at the end, after --depth=1.

It's also possible that was the last URL in your list and it stopped at the end. If you want to re-try archiving you should run this instead:

    archivebox update
Post reply on HN