Earlier quoted context omitted.
What about personal caches? I know that few people likely keep locally saved Web pages in near line accessibility. But if you only build the tool, I would happily mount the nearly 5 terabytes of ULTRASCSI drives I've got in storage, which were largely Squid cache snapshots, from my office WAN proxy box.
You should add them to IPFS, it sounds ideal for your use case.
I'm more inclined to use Apache Lucene.
Both because of the abundance of people doing similar extraction from comparable data stores, but also because Lucene is a part of the Hitachi data suites which are my business choice.. Allowing I ever return to business.
I mention Hitachi only because by anointing Lucene in a Tier 1 product, if I encounter the need to provide a audit that is relied upon to attest any sensitive data is purged from the cache, I'm sure that it will be acceptable in the format of such a high end system report.
Of course that's window dressing,but my aim is to get big businesses to contribute more old data... just who might aggressively retain data for their own reasons, compliance and security the usual needs,
Well those donors are who need the reassurance of a more vendor.
I wrote above the delightful ideas this prompted me to forget about dinner last night, so I will stop here.
But I cannot help but feel in my gut this is a real and viable project that can snowball and be a huge thing.
Oh, sorry, I misled you, I have to add what just now came to my mind : anything that encourages big companies to give into really old data archives, is just when they are ripe for selling new bulk capacity. The need for refreshing the media is so overlooked by businesses that otherwise have great practices. So you can nudge even the least hungry prospect customers towards your sales team.
Can't a vendor jump on this one?
Edit format