This simply solidifies my opinion that SQLite is one of the Best Tools Ever.
Static torrent website with peer-to-peer queries over BitTorrent on 2M records
51–60 of 110 posts
Re: Static torrent website with peer-to-peer queries over BitTorrent on 2M records
#52Does the lazy loading combined with the torrent mean that you can lose individual rows if nobody has accessed them recently?
Not in this case. Visitors to this site are only pulling data from a torrent, not hosting any data for other visitors.
If you want to help host the torrent then you'd be seeding the whole database by default. Torrents are split into blocks and there is no easy way to choose to only download/seed specific blocks of a single file.
Re: Static torrent website with peer-to-peer queries over BitTorrent on 2M records
#53Yup. Calling it. This is the future, may not seem like it to everyone but this is a part of actually new and extremely useful, passively scalable technology. Imagine (good) unkillable zombie databases- so long as the name of a piece of data is known (its hash).. someone, somewhere, might make it possible for you to answer your query, without even needing to setup a server setup.. that’s _it_! Not to mention it might…
This idea touches on (or rather, even expands on) an idea I've had brewing for a while. For those unfamiliar, there is a programming language called Unison [1] that has a nice feature where each function is identified by a hash of its AST. I often thought a cluster environment, maybe something BEAM-ish, would be an interesting idea where functions are retrievable based on a pair of (`func-name`, `hash`). You could have some kind of execution environment (maybe a lightweight process) that receives a hash identifying the function to execute and some pointer to the data (probably a URL) to act as input.
Given tech like described in the linked article, you could reasonably lookup such a function from a peer-to-peer network.
Re: Static torrent website with peer-to-peer queries over BitTorrent on 2M records
#54Does the lazy loading combined with the torrent mean that you can lose individual rows if nobody has accessed them recently?
I am also curious about this.
One of the flaws that traditional torrents suffer from is if all the downloaders want 1 file out of 10 and thus seed 1/10, the torrent can ever again be fully downloaded if the last seeder with 100% drops off.
I suspect this is not an issue for the demo since the DB seeding is probably being monitored, but as a technique? I could definitely see chunks of a DB not being available due to no one seeding 100%
Re: Static torrent website with peer-to-peer queries over BitTorrent on 2M records
#55Yup. Calling it. This is the future, may not seem like it to everyone but this is a part of actually new and extremely useful, passively scalable technology. Imagine (good) unkillable zombie databases- so long as the name of a piece of data is known (its hash).. someone, somewhere, might make it possible for you to answer your query, without even needing to setup a server setup.. that’s _it_! Not to mention it might…
The media control barons have spent the past decade building legal strategies for taking down promising services. Personally I also feel like this is the future, but only in a future where we dismantle their stranglehold on culture.
Anyone who shares content (whether a file or a byte) exposes themselves in one way or another and can be targeted. Anyone else remember the “good old days” when torrent sites stood up in the face of the law and yelled “we host nothing illegal because we host nothing!” and they were right. Then the law firms started joining the swarms to collect IP addresses of everyone that shared with them.
Now they rarely bother trying to sue IP addresses. The legal ability is still there, it just turns out it’s not worth the cost to sometimes collect a few hundred dollars from someone who can’t afford it.
When we talk about transformative tech, we have to immediately consider the “standard scenarios” because being blind to them is how most technology trips and falls. Pirating (especially live PPV events), illegal porn, utterly violent videos, information on making weapons, letting children access same and/or mixing them in with adults in unsafe and unsupervised spaces, openly anti-government statements in countries where they make people disappear, and so-on.
These things happen. And they happen faster and more intensely than most programmers can handle. The only way to keep tools like this viable is to build the tech with those in mind.
Re: Static torrent website with peer-to-peer queries over BitTorrent on 2M records
#56My biggest issue with storing data inside the browser is the lack of tools to set up filter rules for clearing or explicitly not clearing website-specific application data.
Looking at Chrome's Inspector, Application Storage has "Local Storage", "Session Storage", "IndexedDB", "WebSQL" and "Cookies".
If I decide to clear all this stored data stored for all the pages with one button press in the settings, how can I be sure I'm excluding something important? Or if I install a new extension which helps me manage privacy, how can I be sure that it won't delete all this data because I forgot to add a site to its whitelist?
Re: Static torrent website with peer-to-peer queries over BitTorrent on 2M records
#57Earlier quoted context omitted.
Unfortunately this is still killable. Someone needs to periodically publish up to date databases using some outside protocol. Also if you participate in the public DHT, then your participation is public and can have consequences on your real-life legal entity. If you want to use TOR and onion sites, I don't think this really adds anything to those. I think this just helps you optionaly crowdsource bandwidth.
With a simple protocol on top it can be quite resilient. The database is updated only by a group of 3 admins, who perform the update once a week. When one admin initiates an update, the other two ask him the passphrase. If the answer is "tomato" (means "I'm compromised"), the update is rejected and they find another admin. Regular users are generally safe: the laws dont punish downloaders, only uploaders. You can exp…
> Regular users are generally safe: the laws dont punish downloaders, only uploaders.
Are you sure about that. The downloader is the one making the copy. The uploader is merely inducing the creation of said copy.
Yes, it's easier/more efficient to go after the uploader, but that doesn't immunize the downloader.
> Don't forget that the admins can be in different countries, making their legal pursuit a nightmare.
The US govt does multi-country activity all the time.
Re: Static torrent website with peer-to-peer queries over BitTorrent on 2M records
#58Is the demo down for anyone else? It's stuck loading. Chrome on MacOS
The site and search eventually work, but the torrents themselves do not download and do not appear to have all necessary magnet information in the links. So the downloads just hang there in the torrent client.
Re: Static torrent website with peer-to-peer queries over BitTorrent on 2M records
#59Does the lazy loading combined with the torrent mean that you can lose individual rows if nobody has accessed them recently?
Pretty sure. Ideally there are a few always-on seeders for every db.
Re: Static torrent website with peer-to-peer queries over BitTorrent on 2M records
#60Earlier quoted context omitted.
I know everyone is fed up with the cliche of web3, but having done some side projects with web3 (as in npm i web3) and ipfs, the posted article here is very exciting. This posted link could be another piece of the jigsaw (Solidity, IPFS, IPNS, ...) that I think will come together to make interesting apps in the future. Solidity doesn't mean necessarily spending $100 to make a function call - there are other chains, o…
Even as someone who profited well from Uniswap's initial token offering I still have to say that "swapping tokens" is an entirely useless functionality. Gambling, trading, scamming, and laundering are the only realized uses of crypto I have seen since I became involved with the ecosystem a decade ago. The fatal flaw of the decentralization argument is that humans are involved. People want to be able to negotiate char…
> Gambling, trading, scamming, and laundering are the only
You just described any transactional medium, including cash. Prove me wrong.