A search engine for searching books in the Z-Library index on the IPFS network • Hacker News Board

Re: A search engine for searching books in the Z-Library index on the IPFS network

#171

Post by mhluongo » Fri, Dec 30, 2022, 3:08 PM UTC

1vuio0pswjnm7 wrote: post #122

oldgregg wrote: post #26

Earlier quoted context omitted.

Honestly I think Consensys/IPFS/Libp2p is just some corporatized way to derail real P2P and decentralization. Their libraries are total garbage. Lots of complicated code that simply doesn't work. No documentation. I mean look how much IPFS and Libp2p is pumped but IT DOESN'T WORK. IPNS is a joke. All way overengineered crap that does everything but actually nothing. Look at the $$$ and pedigree behind Consensys it's…

Peer-to-peer is too general and too useful to be spoilt by "blockchain". But there is so much money behind this crap it is a formidable virus that can infect any project. ConsenSys is a company formed by Ethereum co-founder. He has sold out to the big banks, in the opinion of some Consensys shareholders. https://finance.yahoo.com/news/consensys-shareholders-readyi... "Falls says he believes both MetaMask and Infura -…

Again, ConsenSys isn't behind IPFS. These are two different things.

> People are now trying to associate IPFS with "Web3". For example, check out this paper published a couple of weeks ago.

IPFS has been used to host dApps for a long time — eg the Uniswap frontend. This isn't a recent thing.

Re: A search engine for searching books in the Z-Library index on the IPFS network

#172

Post by 2color » Fri, Dec 30, 2022, 4:17 PM UTC

oldgregg wrote: post #46

survirtual wrote: post #35

Earlier quoted context omitted.

Anything is possible, but having some limited github interactions with the core team, that seems unlikely. My impression is that they are a passionate group that hit the jackpot a bit prematurely. Their intentions seemed good to me they just have an ungodly amount of financing while perhaps lacking a core vision & understanding of what is at stake. Put another way, seems more like an academic building something rathe…

I certainly wouldn't disparage the talent- no doubt vast majority have good intentions but unfortunately they are being paid handsomely to have their work be rendered ineffective. Gobble up all the best P2P devs and get them focused on work that is largely ineffective. Lots of code and specs but nothing really usable. Code is not very original, they just take other libraries and tweak them. mplex is their version of…

I think that this is a misunderstanding but I understand where this is coming from.

Let me share my 2 cents as the Developer Advocate for IPFS (employed by PL).

You claim that our work is largely ineffective but what are you referring to? The IPFS ecosystem is like a cluster of several ecosystems, e.g. Libp2p, IPLD, IPFS, Multiformats.

> Lots of code and specs but nothing really usable

Some of the libraries may not be as actively maintained as others — this is the nature of open source. But to claim nothing is usable is not accurate — there's a lot of stuff that works pretty reliably.

> Code is not very original

Who cares....

> autonat is their version of stun/ice.

It's actually Autonat, DCUtR, and Circuit Relay V2 which are pretty amazing given that it's like STUN/ICE but decentralized! Anyone in the network might help you hole punch through NAT and from network measurements the success rates average around 70% (https://youtu.be/bzL7Y1wYth8?t=1091). Saying it's just "their version of STUN/ICE) takes away from the main qualitative difference of doing it without centralized coordination servers.

> "what apps use libp2p" you won't find anything because nobody uses it.

Ethereum uses libp2p GossipSub since the beacon chain launched. Berty, Satellite (https://satellite.im/), Status (https://discuss.status.im/t/waku-version-2-pitch/1776#org900...).

Either way, I appreciate your comment, we need to be doing a better job of surfacing some of the great progress being made in the IPFS/Libp2p ecosystem.

Re: A search engine for searching books in the Z-Library index on the IPFS network

#173

Post by 2color » Fri, Dec 30, 2022, 4:27 PM UTC

1vuio0pswjnm7 wrote: post #122

oldgregg wrote: post #26

Earlier quoted context omitted.

Honestly I think Consensys/IPFS/Libp2p is just some corporatized way to derail real P2P and decentralization. Their libraries are total garbage. Lots of complicated code that simply doesn't work. No documentation. I mean look how much IPFS and Libp2p is pumped but IT DOESN'T WORK. IPNS is a joke. All way overengineered crap that does everything but actually nothing. Look at the $$$ and pedigree behind Consensys it's…

Peer-to-peer is too general and too useful to be spoilt by "blockchain". But there is so much money behind this crap it is a formidable virus that can infect any project. ConsenSys is a company formed by Ethereum co-founder. He has sold out to the big banks, in the opinion of some Consensys shareholders. https://finance.yahoo.com/news/consensys-shareholders-readyi... "Falls says he believes both MetaMask and Infura -…

> IPFS now uses base58btc exclusively

That's blatantly wrong. IPFS supports 25 different base representations (https://github.com/multiformats/multibase/blob/master/multib...).

In fact, recently, two community members decided to implement a new base encoding with emojis for fun:

https://cid.ipfs.tech/#%F0%9F%9A%80%F0%9F%AA%90%E2%AD%90%F0%...

https://github.com/multiformats/multihash supports at the very least SHA1 SHA2-256 SHA2-512 SHA3/Keccak Blake2b-256/Blake2b-512/Blake2s-128/Blake2s-256 Blake3 and Strobe. Hashes in IPFS are being standardised through the IETF and W3C https://www.ietf.org/id/draft-multiformats-multihash-05.html.

If you need rhash, you are welcome to submit a PR! We also have a grants program you can use to be rewarded for this.

Re: A search engine for searching books in the Z-Library index on the IPFS network

#174

Post by 2color » Fri, Dec 30, 2022, 4:35 PM UTC

KirillPanov wrote: post #79

survirtual wrote: post #14

Earlier quoted context omitted.

I’ve dived deep into IPFS and built several prototypes on top of it. It ended up not being performant enough for me, and that was after heavily modifying the codebase so that it was true p2p browser & server (their webrtc transport had a lot of issues and they didn’t seem too interested in it, but my needs required it as a backbone). The security was also a concern, and the scaling had issues. Pinning millions of sma…

Same experience here. The #1 lesson they failed to learn from Bittorrent and Tox: your DHT must be sessionless . QUIC is not a solution to this particular problem. Although QUIC uses UDP, it still creates and manages sessions. This is incompatible with decentralized scaling. At the scale of Bittorrent and DNS you cannot manage this kind of session state without centralization.

IPFS encompasses more than just a DHT: - representing content addressed data with hashes in CIDs - Discovering them (via the DHT) - Transfering (Bitswap, HTTP, Graphsync, sneakernet and more in development)

I'm not 100% certain what you mean by the DHT being sessionless.

> At the scale of Bittorrent and DNS you cannot manage this kind of session state without centralization.

You don't have to route content via the public DHT. Depending on what kind of data you're publishing, you can use a private DHT swarm, or delegated routing with HTTP.

> This is incompatible with decentralized scaling.

Re: A search engine for searching books in the Z-Library index on the IPFS network

#175

Post by RunSet » Fri, Dec 30, 2022, 4:40 PM UTC

1vuio0pswjnm7 wrote: post #122

oldgregg wrote: post #26

Earlier quoted context omitted.

Honestly I think Consensys/IPFS/Libp2p is just some corporatized way to derail real P2P and decentralization. Their libraries are total garbage. Lots of complicated code that simply doesn't work. No documentation. I mean look how much IPFS and Libp2p is pumped but IT DOESN'T WORK. IPNS is a joke. All way overengineered crap that does everything but actually nothing. Look at the $$$ and pedigree behind Consensys it's…

Peer-to-peer is too general and too useful to be spoilt by "blockchain". But there is so much money behind this crap it is a formidable virus that can infect any project. ConsenSys is a company formed by Ethereum co-founder. He has sold out to the big banks, in the opinion of some Consensys shareholders. https://finance.yahoo.com/news/consensys-shareholders-readyi... "Falls says he believes both MetaMask and Infura -…

> IPFS now uses base58btc exclusively. That's "btc" as in Bitcoin.

It is true that Base58 is named for its original implemention in bitcoin but other than the name it has nothing to do with bitcoin.

The difference from Base64: It excludes non-alphanumeric characters and visually similar characters (I, l; O, 0) to make the output more human-friendly[0].

The problem it attempts to solve: Making bitcoin addresses more human-readable. That is likely also why IPFS uses it.

[0] https://en.bitcoinwiki.org/wiki/Base58#How_it_works.3F

Re: A search engine for searching books in the Z-Library index on the IPFS network

#176

Post by 0ct4via » Fri, Dec 30, 2022, 5:04 PM UTC

jraby3 wrote: post #145

I got an error when visiting zlib.zu1k.com/. Error code: 1020 Ray ID: 7819e5389c898e4e Country: IL Data center: tlv03 IP: 85.65.187.66 Timestamp: 2022-12-30 09:54:40 UTC

It's possible they were "hugged" enough to instigate a firewall rule dramatically cutting down allowed visitors, which would explain the Error 1020.

Might be worth reaching out to the site's creator -- info taken from their site [https://zu1k-com/about/]:

"If you have any questions to communicate with me, the preferred way is to comment directly in the comment area of the corresponding blog post, and I will receive an email notification

In addition, you can also reach me in the following ways:

Email: i@zu1k.com

Twitter: @zu1k_lv

GitHub: @zu1k

Keybase: @zu1k

Matrix: @zu1k:mozilla.org

My PGP public key can be obtained at https://pgp.zu1k.com or hkps: //keyserver.ubuntu.com

( 2A65 F6F3 1EDA D922 D7E7 E97B AE38 1A8F B1EF 2CC8)"

Re: A search engine for searching books in the Z-Library index on the IPFS network

#177

Post by TEP_Kim_Il_Sung » Fri, Dec 30, 2022, 5:38 PM UTC

mhluongo wrote: post #95

oldgregg wrote: post #26

Earlier quoted context omitted.

Honestly I think Consensys/IPFS/Libp2p is just some corporatized way to derail real P2P and decentralization. Their libraries are total garbage. Lots of complicated code that simply doesn't work. No documentation. I mean look how much IPFS and Libp2p is pumped but IT DOESN'T WORK. IPNS is a joke. All way overengineered crap that does everything but actually nothing. Look at the $$$ and pedigree behind Consensys it's…

You're confusing ConsenSys and Protocol Labs. ConsenSys was founded by Joe Lubin, and is the corporate entity behind MetaMask, Infura, etc. It counts MasterCard and JP Morgan as investors. Protocol Labs was founded by Juan Benet and is the primary sponsor of IPFS and the dev team behind Filecoin. They're funded primarily from the private token sales of FIL. Agree with the sibling comment that the focus on modularity…

ConsenSys is to IPFS as Blockstream is to BTC

Re: A search engine for searching books in the Z-Library index on the IPFS network

#178

Post by LarryMullins » Fri, Dec 30, 2022, 6:07 PM UTC

jerojero wrote: post #56

Something that has always bothered me about z-lib and libgen and so on is the impossibility of being able to tell quality from the different uploads. Sometimes the epubs are very good, with a good cover and metadata as well as chapters and no weird aberrations; other times, they're unreadable. But the only thing you have to judge quality is size and general description of the upload which in this website it's been re…

The books are free, if you download the wrong version of a book you can simply try another and it costs you no more than a minute of your time.

Do you know how it worked before we had the internet for books? You had to go to a library yourself, which may have taken an hour of your day just in the journey, then hope the library had the book you wanted. If not, you may have to wait weeks for another library in the system to mail the book to your library.

Re: A search engine for searching books in the Z-Library index on the IPFS network

#179

Post by LarryMullins » Fri, Dec 30, 2022, 6:19 PM UTC

cratermoon wrote: post #40

A question about z-lib, libgen, and regular libraries Suppose I check an e-book out of my local library. Because Reasons, the library doesn't "own" unlimited "copies" of the book, so each "copy" can only be checked out to one patron at a time and if there are enough holds then a patron can't renew a checked out book. In short, ebooks in libraries are just like regular paper books, except to you don't have leave your…

Of the past twenty ebooks I've downloaded, 18 have dead authors and only one is alive.

Unless you focus on the very latest releases, most authors are already dead. If you round up just about any list of great works from the 20th century, you'll find that most of those authors are dead and have been for years. If you like 19th century books, all of those authors are dead.

Re: A search engine for searching books in the Z-Library index on the IPFS network

#180

Post by LarryMullins » Fri, Dec 30, 2022, 6:38 PM UTC

leobg wrote: post #134

o_____________o wrote: post #48

Earlier quoted context omitted.

What's the best off the shelf solution you've found? I've been trying a few different apps to read my queued articles to me, but all of the voices are awful.

And best open source TTS engine I found so far is Coqui TTS. Demos here: https://mbarnig.github.io/TTS-Models-Comparison/

I've tried Coqui for generating ebooks and found the edge-cases to be too severe for this to be usable; Coqui has a nasty habit of having strokes, for instance "hello" may become "hellooooooOOoooooooooooOOOOoOOOOOoOOOOOooooooOOOOOOOOOOOoooooooooOOOOOOOooo...." With some manipulation of the input (like adding punctuation) you can get around this, but if you're trying to automate this for a whole book then I think you'll need a lot of trial and error before you get the input massaging heuristics right.

The best TTS system for generating ebooks (that I have used so far) is MacOS's TTS. The process is fairly straight forward; first break the book into sentences using the standard heuristics. Run each of those sentences through TTS to generate an audio file for that sentence, then create a subtitle file with the text of the sentence. Then stitch them all together with ffmpeg, optionally adding a dummy video track if your media player of choice needs one to display subtitles (mpv does not, but some do.) Now you have a subtitled "audiobook."

MacOS's TTS has a robotic quality to it, but it generally works and you can become accustomed to it fairly easily if you give it a chance.