"Native Data Deduplication" not supported in CBZ/CBR? But those are just ZIP/RAR, which are compression formats, deduplication is their whole deal...?
Libbbf: Bound Book Format, A high-performance container for comics and manga
21–30 of 66 posts
Re: Libbbf: Bound Book Format, A high-performance container for comics and manga
#22Thinking more about this: ZIP files can be set up to have the data on whatever alignment of one's choosing (as noted in the reddit thread). Integrity checks can be done in parallel by doing them in parallel. mmap is possible just by not using zip compression.
The aspect of integrity checking speed in a saturated context (N workers, regardless if its multiple workers per file, or a worker per file), CRC32(C) seems to be nearly twice as fast https://btrfs.readthedocs.io/en/latest/Checksumming.html
ZIP can also support arbitrary metadata.
I think this could have all been backported to ZIP files themselves
Re: Libbbf: Bound Book Format, A high-performance container for comics and manga
#23https://www.reddit.com/r/selfhosted/comments/1qi64pr/i_got_i...
Maybe you should quote the full title of that post:
"I got into an argument on Discord about how inefficient CBR/CBZ is, so I wrote a new file format. It's 100x faster than CBZ."
It has some charts, notes and comments
Here's the old.reddit link: https://old.reddit.com/r/selfhosted/comments/1qi64pr/i_got_i...
Re: Libbbf: Bound Book Format, A high-performance container for comics and manga
#24Honest question, something I don't understand, if you use DirectStorage to move images directly to the GPU (I assume into the VRAM) where the decoding take place? directly on the GPU? Can GPU decode PNG? it is very unfriendly format for GPU as far as I know
From the readme:
> Note: DirectStorage isn't avaliable for images yet (as far as I know), but I've made sure to accomodate such a thing in the future with this format.
So the whole DirectStorage thing is just a nothingburger. The author glosses over the fact that decoding images on GPU is not possible (or at least very impractical).
Re: Libbbf: Bound Book Format, A high-performance container for comics and manga
#25Earlier quoted context omitted.
Zip also has per-asset checksums, contrary to the comparison table. And what's the point of aligning the files to be "DirectStorage-ready" if they're going to be JPEGs, a format that, as far as I know, DirectStorage doesn't understand? And the author says it's a problem that "Metadata isn't native to CBZ, you have to use a ComicInfo.xml file.", but... that's not a problem at all? The whole thing makes no sense.
It makes no sense because it's some degree of AI slop: https://reddit.com/r/selfhosted/comments/1qi64pr/i_got_into_... Note that he doesn't quite say, when asked pointblank how much AI he used in his erroneous microbenchmarking, that he didn't use AI: https://reddit.com/r/selfhosted/comments/1qi64pr/i_got_into_... Which explains all of it. Kudos to /u/teraflop, for having infinitely more patience with this than I wou…
I'm a moderator for a decently large programming subreddit, and I'd estimate about half the project submissions now being obvious slop. You get a very good nose for sniffing that stuff out after a while, though it can be frustrating when you can't really convince other people beyond going "trust me, it's slop".
Re: Libbbf: Bound Book Format, A high-performance container for comics and manga
#26> Footer indexed So, like ZIP? > Uses XXH3 for integrity checks I don’t think XXH3 is suitable for that purpose. It’s not cryptographically secure and designed mostly for stuff like hash tables (e.g. relatively small data).
> It’s not cryptographically secure
Neither is CRC32. I'm pretty sure xxhash is a straight upgrade compared to CRC32.
Re: Libbbf: Bound Book Format, A high-performance container for comics and manga
#27I use CBZ to archive both physical and digital comic books so I was interested in the idea of an improved container format, but the claimed improvements here don't make sense. --- For example they make a big deal about each archive entry being aligned to a 4 KiB boundary "allowing for DirectStorage transfers directly from disk to GPU memory", but the pages within a CBZ are going to be encoded (JPEG/PNG/etc) rather th…
> The most charitable interpretation is that this whole project (supposed problems with CBZ, the readme, the code) is the output of an LLM.
Do LLMs perform de/serialization by casting C structs to char-pointers? I would've expected that to have been trained out of them. (Which is to say: lots of it is clearly LLM-generated, but at least some of the code might be human.)
Anyway, I hope that the person who published this can take all the responses constructively. I know I'd feel awful if I was getting so much negative feedback.
Re: Libbbf: Bound Book Format, A high-performance container for comics and manga
#28Why are the metadata blocks the way they are? I see you used pack directives but there already are plenty of padding and reserved bits. A 19 byte header just seems wrong.
https://github.com/ef1500/libbbf/blob/b3ff5cb83d5ef1d841eca1...
Re: Libbbf: Bound Book Format, A high-performance container for comics and manga
#29This feels like the wrong end to optimize. Zip is plenty of fast, especially when it comes to a few hundred pages of a comic. Meanwhile the image decoding can take a while when you want to have a quick thumbnail overview showing all those hundred pages at once. No comic/ebook software I have ever touched as managed to match the responsiveness of an actual book where you can flip through those hundreds of pages in a second with zero loading time, despite it being somewhat trivial to implement when you generate the necessary thumbnail/image-pyramid data first.
A multi-resolution image format would make more sense than optimizing the archive format. There would also be room for additional features like multi-language support, searchable text, … that the current "jpg in a zip" doesn't handle (though one might end up reinventing DJVU here).
Re: Libbbf: Bound Book Format, A high-performance container for comics and manga
#30Earlier quoted context omitted.
It makes no sense because it's some degree of AI slop: https://reddit.com/r/selfhosted/comments/1qi64pr/i_got_into_... Note that he doesn't quite say, when asked pointblank how much AI he used in his erroneous microbenchmarking, that he didn't use AI: https://reddit.com/r/selfhosted/comments/1qi64pr/i_got_into_... Which explains all of it. Kudos to /u/teraflop, for having infinitely more patience with this than I wou…
That whole subreddit has unfortunately become inundated with AI slop. It used to be a decent resource to learn about what services people were self hosting. But now, many posts are variations of, “I’ve made this huge complicated app in an afternoon please install it on your server”. I’ve even seen a vibe-coded password manager posted there. Reputable alternatives to the software posted there exist a a huge amount of…
3 days ago the rules changed that vibe coded stuff is only allowed on Fridays.
https://old.reddit.com/r/selfhosted/comments/1qfp2t0/mod_ann...