Live data from Hacker News

SeaweedFS fast distributed storage system for blobs, objects, files and datalake

github.com

21–30 of 128 posts

Re: SeaweedFS fast distributed storage system for blobs, objects, files and datalake

#21
This sounds like what Microsoft has tried but failed to do in numerous iterations for two decades: OFS (Cairo, unreleased predecessor to Windows 95), Storage+ (SQL Server 7.0), RFS (SQL Server 2000), Exchange Webstore, Outlook LIS, WinFS, and finally Microsoft Semantic Engine.

All projects were either cancelled, features cut, or officially left in limbo.

It's a pretty remarkable piece of Microsoft history as it has been there on the sidelines since roughly post-Windows 3.11. The reason they returned to it so often was in part because Bill Gates loved the idea of a more high level object storage that, like this, bridges the gap between files and databases.

He would probably have loved this kind of technology part of Windows -- and indeed in 2013, he cited the failure of WinFS as his greatest disappointment at Microsoft, that it was ahead its time and that it would re-emerge.

Re: SeaweedFS fast distributed storage system for blobs, objects, files and datalake

#22

SeaweedFS does the thing: I've used it to store billions of medium-sized XML documents, image thumbnails, PDF files, etc. It fills the gap between "databases" (broadly defined; maybe you can do few-tens-KByte docs but stretching things) and "filesystems" (hard/inefficient in reality to push beyond tens/hundreds of millions of objects; yes I know it is possible with tuning, etc, but SeaweedFS is better-suited). The do…

I was quite surprised to discover that minio is one file per object. Having read some papers about object stores, this is definitely not what I expected.

Re: SeaweedFS fast distributed storage system for blobs, objects, files and datalake

#23
post #2

I was asking around in my network after experience with self hosting S3 like solutions. One serious user of SeaweedFS recommended looking into min.io instead. Another serious user of min.io recommend looking into SeaweedFS instead…

A serious user of both suggested to use iroh instead

Re: SeaweedFS fast distributed storage system for blobs, objects, files and datalake

#24

SeaweedFS does the thing: I've used it to store billions of medium-sized XML documents, image thumbnails, PDF files, etc. It fills the gap between "databases" (broadly defined; maybe you can do few-tens-KByte docs but stretching things) and "filesystems" (hard/inefficient in reality to push beyond tens/hundreds of millions of objects; yes I know it is possible with tuning, etc, but SeaweedFS is better-suited). The do…

GarageS3 is a nice middle ground, it is not file on disk per object but it's simpler than SeaweedFS as well.

https://garagehq.deuxfleurs.fr/

Re: SeaweedFS fast distributed storage system for blobs, objects, files and datalake

#25
post #2

I was asking around in my network after experience with self hosting S3 like solutions. One serious user of SeaweedFS recommended looking into min.io instead. Another serious user of min.io recommend looking into SeaweedFS instead…

Take a look at GarageS3, it's a niceoption for "just an S3 server" for self hosting.

https://garagehq.deuxfleurs.fr/

I use it for self hosting.

Re: SeaweedFS fast distributed storage system for blobs, objects, files and datalake

#26
post #4

We tested both SeaweedFS and Min.io for cheaply (HDD) storing > 100TB of audio data. Seaweed had much better performance for our use case.

Forgive my ignorance but why is this preferable to a big ZFS pool?

I could be wrong here, but I believe this (ceph, et al) is the answer to the question: > """But what if I don't have a JBOD of 6x18TB hard drives with good amount of ECC RAM for ZFS? What if I have 3 raspberry pi 4's, at different houses with 3x 12TB externals on them, and 2 other computers with 2x 4TB externals on them, and I want to use that all together with some redundancy/error checking?" That would give (3x3x12)+(2x2x4)=124 TB of storage, vs 108TB in the ZFS single box case (of raw storage).

If you could figure out the distributed part (and inconsistency in disk size and such), then this is a very nice system to have.

Re: SeaweedFS fast distributed storage system for blobs, objects, files and datalake

#27
Have used SeaweedFS to store billions of thumbnails. The tooling is a bit clunky, but it mostly works. The performance is very good for small-ish objects (memory usage + latency), and latency remains consistently good into 99.9 percentiles. We had some issues with data loss and downtime, but that was mostly our own fault.

Re: SeaweedFS fast distributed storage system for blobs, objects, files and datalake

#28
post #23
post #2

I was asking around in my network after experience with self hosting S3 like solutions. One serious user of SeaweedFS recommended looking into min.io instead. Another serious user of min.io recommend looking into SeaweedFS instead…

A serious user of both suggested to use iroh instead

If you're talking about this https://github.com/n0-computer/iroh ... Iroh is a p2p file syncing protocol. That's not even close to the same wheelhouse as SeaweedFS. What was their rationale for recommending it?

Re: SeaweedFS fast distributed storage system for blobs, objects, files and datalake

#30
post #26

Earlier quoted context omitted.

Forgive my ignorance but why is this preferable to a big ZFS pool?

I could be wrong here, but I believe this (ceph, et al) is the answer to the question: > """But what if I don't have a JBOD of 6x18TB hard drives with good amount of ECC RAM for ZFS? What if I have 3 raspberry pi 4's, at different houses with 3x 12TB externals on them, and 2 other computers with 2x 4TB externals on them, and I want to use that all together with some redundancy/error checking?" That would give (3x3x12…

"What if I have 3 raspberry pi 4's [...] with no significant performance requirement?"

Ceph is nice, but performance is lackluster on anything but a proper cluster (pun intended).

It's also somewhat heavyweight. I ran ZFS over iSCSI with four RPis serving the iSCSI targets via SATA-USB. It was network limited mostly. The advantage of that is that you can take the same disks and plug them all into a single host and import the pool directly (ie not via iSCSI), if needed.

Post reply on HN