81–90 of 166 posts
I have documented that here: https://darkstar.github.io/2022/02/07/k3s-on-raspberrypi-at-...
Note that this was a rather old version and some things have probably changed compared to now, so YMMV, but it might still provide a good reference for those who want to try
Have you (and I wonder the same about archive.org) considered making a Merkle tree of the data that gets archived? Since data (including photos and videos) are getting easier to fake, it may be nice to have a provable record that at least a certain version of the data existed at a certain time. It would be most useful in case of some sort of oppressive regime down the line that wants to edit history. You'd want to pu…
Not just all that nonsense, but also it makes a lot of sense to share just the parts from a website that matter like a single video etc without having to download an entire archive or the rest of the site
We try to make that easy by allowing ppl to select one or more specific archivebox extractors when adding, so you don t have to archive everything every time.
Makes it more useful for scraping in a pipeline with some other tools.
@nikisweeting ArchiveBox is awesome and we'd really love it to be more awesome. And sustainable! I've posted issues and PRs for showstopper issues that took months to get merged in: https://github.com/ArchiveBox/ArchiveBox/issues/991 https://github.com/ArchiveBox/ArchiveBox/pull/1026 You have the opportunity for the community to lean in on ArchiveBox. I understand it's hard to do everything as a solo dev, we've seen…
Let chat more. I'm almost ready to raise some seed money, hire a second staff dev or find a cofounder, and I'm looking for people that care deeply about the space. It's only been during the last few months that I decided to go all in on the project, so this is still just the first few pages of a new chapter in the project's history. (I should also mention that if you're a commercial entity relying on ArchiveBox, you…
I'd love to see blockchain proof/notary support. The ability to say "content matching this hash existed at this time.
I'm exceptionally busy now but that being said, I may choose to contribute nonetheless.
I'd love to connect directly, and will connect to the Zulip instance later.
If we align on values, I may be able to connect you with some cash. People often call me an "anarchist" or "libertarian", though I'm just me, not labels necessary.
Earlier quoted context omitted.
The /cli/list endpoint is the search endpoint you're looking for. It provides FTS but I can make it clearer in the docs, thanks for the tip. As for the AI stuff don't worry, none of it is touching core, it's all in an optional community plugin only for those who want it. I'm not personally a huge AI person but I have clients who are already using it and getting massive value from it, so it's worth mentioning. (They'r…
I have been using ArchiveBox recently and love it. About search, one thing I haven't yet figured out how to do easily is to plug it to my SearXNG instance as they only seem to support Elasticsearch, Meilisearch or Solr [0] So this new plugin architecture will allow for a meilisearch plugin I guess (with relevancy ranking). - [0] https://docs.searxng.org/dev/engines/offline/search-indexer-...
We already provide Sonic, ripgrep, and SQLiteFTS as plugins, so adding something like Solr should be straightforward.
Check out the existing plugins to see how it's done: https://github.com/ArchiveBox/ArchiveBox/pull/1534/files?fil...
archivebox/plugins_search/sonic/*
Some time ago I installed ArchiveBox on a RaspberryPi 4 running k3s (a lightweight Kubernetes distro). I have documented that here: https://darkstar.github.io/2022/02/07/k3s-on-raspberrypi-at-... Note that this was a rather old version and some things have probably changed compared to now, so YMMV, but it might still provide a good reference for those who want to try
Happy to report that most of the quirks you cover have been improved:
- uid 999 is no longer just enforced, you can pass any PUID:GUID now (like Linuxserver.io containers)
- it now accepts ADMIN_USERNAME + ADMIN_PASSWORD env vars to create an initial admin user on first start without having to exec
- archivebox/archivebox:latest is 0.7.2 (yearly stable release) and :dev is the 0.8.x pre-release updated daily. All Images are all amd64 & arm64 compatible.
- singlefile and sonic are now included in all images & available on all platforms amd64/arm64
As someone who was archiving a doomed website earlier today using wget, I was reminded that really need to get ArchiveBox working... I used to rely on my Pinboard subscription, but apparently archive exports haven't worked for years, so those days are over.
@nikisweeting ArchiveBox is awesome and we'd really love it to be more awesome. And sustainable! I've posted issues and PRs for showstopper issues that took months to get merged in: https://github.com/ArchiveBox/ArchiveBox/issues/991 https://github.com/ArchiveBox/ArchiveBox/pull/1026 You have the opportunity for the community to lean in on ArchiveBox. I understand it's hard to do everything as a solo dev, we've seen…
Let chat more. I'm almost ready to raise some seed money, hire a second staff dev or find a cofounder, and I'm looking for people that care deeply about the space. It's only been during the last few months that I decided to go all in on the project, so this is still just the first few pages of a new chapter in the project's history. (I should also mention that if you're a commercial entity relying on ArchiveBox, you…
- backing up an entire page
Yes, it is hard. Yes, for non-pure html pages is extra kind of painful, but that would honestly making archivebox go from nice to have to.. yes, I have an actual archive I can use when stuff goes down.
Some time ago I installed ArchiveBox on a RaspberryPi 4 running k3s (a lightweight Kubernetes distro). I have documented that here: https://darkstar.github.io/2022/02/07/k3s-on-raspberrypi-at-... Note that this was a rather old version and some things have probably changed compared to now, so YMMV, but it might still provide a good reference for those who want to try
Thanks for making that tutorial! Happy to report that most of the quirks you cover have been improved: - uid 999 is no longer just enforced, you can pass any PUID:GUID now (like Linuxserver.io containers) - it now accepts ADMIN_USERNAME + ADMIN_PASSWORD env vars to create an initial admin user on first start without having to exec - archivebox/archivebox:latest is 0.7.2 (yearly stable release) and :dev is the 0.8.x p…
Earlier quoted context omitted.
I can accept the other issues, but archivebox needs be private and secure by default. Sending everything to archive.org is bad default value and it erodes a certain level of trust in the project. Requiring "several important changes and security considerations" just makes a non-starter. The default settings should be "safe" for the default user, because as you mentioned in that post, 90% of users are never going to c…
> The default settings should be "safe" for the default user, I 100% agree, but because private archiving is doable but NOT 100% safe yet I cant make that mode the default. The difficult reality currently is that archiving anything non-public is not simple to make safe. Every capture will contain reflected session cookies, usernames, and PII, and other sensitive content. People don't understand that this means if the…
I've been very impressed by all of your responses in here, but that one in particular shows empathy, compassion, and a deep deep subject matter expertise.