Related:
Garage, our self-hosted distributed object storage solution - https://news.ycombinator.com/item?id=30256753 - Feb 2022 (130 comments)
Garage: An open-source distributed object storage service
61–70 of 81 posts
Re: Garage: An open-source distributed object storage service
#62Is there any OSS implementation that is not AGPL?
Triton Manta is MPL
Re: Garage: An open-source distributed object storage service
#63Earlier quoted context omitted.
Yes. Specially S3 which is way overpriced. I have setup already a multi-region minio cluster at the cost of $0.0039/GB. The only "cloud" solution that could get closer to that is Storj. [0] [0]: https://www.storj.io/
Isn't storj using a cryptocurrency?
You pay for the service in cash if you want. The token is to pay out the nodes storing it.
Re: Garage: An open-source distributed object storage service
#64Just finished installing it on my OpenIndiana NAS to replace Minio. Biggest difference so far is that Minio is just files on disk, Garage chunks all files and has a metadata db. Minios listing operations were horribly slow, still have to see if Garage resolves that.
> Biggest difference so far is that Minio is just files on disk Minio _was_ just files on disk. They don't support that mode anymore since 2022-10-29 (see the big yellow warning box at [1]). [1] https://min.io/docs/minio/linux/operations/install-deploy-ma...
Yeah, I have actually frozen Minio at this older version in my stack, as "just files on disk" was the primary feature that drew me to it. I don't want my data locked into some custom format. I'd be willing to bet that ZFS will still be supported in 20 years, but I would not make the same bet about Minio.
For the same reason, it looks like Garage is not an option for my use case.
Re: Garage: An open-source distributed object storage service
#65Earlier quoted context omitted.
Why would that matter? Apps using its api don't need to be AGPL
First of all, it isn't clear this is the case. But this entire comment thread on the AGPL misses the mark. It doesn't matter that the AGPL hasn't been tested in court or what fine grained distinctions you apply to the license or what the AGPL intends. No company in their right mind would risk using software licensed under the AGPL because the result of being wrong would be catastrophic. The legal advice to be skeptic…
Or, you could use AGPL software and license it under AGPL as well. Considering most money today is made in the hosting and servicing and not selling license, I don't see why you would bother caring about old fashioned ways.
Re: Garage: An open-source distributed object storage service
#66Is there any OSS implementation that is not AGPL?
Triton Manta is MPL https://github.com/TritonDataCenter/manta
Are you sure this offers the same S3 compatible API? It sure does look like it rolled its own API[1], which I guess is fine so long as you're entirely in the Triton ecosystem, but makes reusing existing software harder than necessary without that compatibility layer. And that's not even getting into this absolutely mess: https://github.com/TritonDataCenter/manta#repositories it reminds me of the "Microservices" video come to life
1: https://github.com/TritonDataCenter/manta/blob/master/docs/u...
Re: Garage: An open-source distributed object storage service
#67Earlier quoted context omitted.
First of all, it isn't clear this is the case. But this entire comment thread on the AGPL misses the mark. It doesn't matter that the AGPL hasn't been tested in court or what fine grained distinctions you apply to the license or what the AGPL intends. No company in their right mind would risk using software licensed under the AGPL because the result of being wrong would be catastrophic. The legal advice to be skeptic…
Or, you could use AGPL software and license it under AGPL as well. Considering most money today is made in the hosting and servicing and not selling license, I don't see why you would bother caring about old fashioned ways.
You're right, Google should drop their policy and open source the whole thing, since it's their data and GCP services that are the crown jewels, right? https://opensource.google/documentation/reference/using/agpl...
Re: Garage: An open-source distributed object storage service
#68Earlier quoted context omitted.
> I'd kind of expect most blob storage solutions to use abstractions other than just the file system, or at least consider doing so. Honestly, I'd expect the exact opposite. Filesystems are really good at storing files. Why not leverage all that work? > I recently built a system to handle millions of documents as a proof of concept and when I was testing it with 10 million files, the server ran out of inodes, before…
> Honestly, I'd expect the exact opposite. Filesystems are really good at storing files. Why not leverage all that work? There are lots of different file systems out there and you won't always get a say in what your cloud vendor has on offer. However, if you can launch a container on the system that does an abstraction on top of the file system, takes its best parts and makes up for any shortcomings it might have in…
To put it pretty bluntly: you were off the rails at "getting a VPS from provider X doesn't mean that they will". You're talking in terms of not having a custom kernel, and that's just the wrong layer of abstraction if we're talking about "cloud"; this whole discussion is really about VM and colo levels of abstraction anyway ("Cloud" advice would be "Just use your vendor's S3 blobstore").
Base Ubuntu has xfs support. If your VPS provider won't run plain old Ubuntu with some cloudinit stuff, get a new VPS provider.
Re: Garage: An open-source distributed object storage service
#69Earlier quoted context omitted.
> Biggest difference so far is that Minio is just files on disk Minio _was_ just files on disk. They don't support that mode anymore since 2022-10-29 (see the big yellow warning box at [1]). [1] https://min.io/docs/minio/linux/operations/install-deploy-ma...
Yeah, I have actually frozen Minio at this older version in my stack, as "just files on disk" was the primary feature that drew me to it. I don't want my data locked into some custom format. I'd be willing to bet that ZFS will still be supported in 20 years, but I would not make the same bet about Minio. For the same reason, it looks like Garage is not an option for my use case.
I use this "just files on disk" feature too, I also use ZFS.
I have a bunch of crawlers uploading data to AWS S3, but S3 is too expensive, I replaced S3 with MinIO.
MinIO stores plain files on disk, which makes it a lot easier to access my data. I can read files without calling MinIO APIs, the speed is super fast.
By the way, which old version are you using? I'm using RELEASE.2022-04-26T01-20-24Z
Re: Garage: An open-source distributed object storage service
#70Earlier quoted context omitted.
> Biggest difference so far is that Minio is just files on disk, Garage chunks all files and has a metadata db. I'd kind of expect most blob storage solutions to use abstractions other than just the file system, or at least consider doing so. I recently built a system to handle millions of documents as a proof of concept and when I was testing it with 10 million files, the server ran out of inodes, before I went over…
> I'd kind of expect most blob storage solutions to use abstractions other than just the file system, or at least consider doing so. Honestly, I'd expect the exact opposite. Filesystems are really good at storing files. Why not leverage all that work? > I recently built a system to handle millions of documents as a proof of concept and when I was testing it with 10 million files, the server ran out of inodes, before…
> Honestly, I'd expect the exact opposite. Filesystems are really good at storing files. Why not leverage all that work?
File systems are optimized for a hierarchical organization on a single machine. However, this kind of organization inhibits storing data in a distributed system because of the links between entries. S3 and similar object stores are a distributed, flat “filesystem”. There’s no relationship between files and there’s no grouping (aside from a virtual one you can simulate but doesn’t really exist). That’s why S3 doesn’t suffer weird directory traversal attacks that bring your file system to a crawl because such potentially expensive operations don’t exist.