Any good alternatives for local development?
seaweedfs: `weed server -s3` is enough to spin up a server locally
MinIO repository is no longer maintained
61–70 of 397 posts
Re: MinIO repository is no longer maintained
#62[dead]
I think the landscape has changed with those hyperscalers outcompeting open-source projects with alternative profit avenues for the money available in the market.
From my experience, Ceph works well, but requires a lot more hardware and dedicated cluster monitoring versus something like more simple like Minio; in my eyes, they have a somewhat different target audience. I can throw Minio into some customer environments as a convenient add-on, which I don't think I could do with Ceph.
Hopefully one of the open-source alternatives to Minio will step in and fill that "lighter" object storage gap.
Re: MinIO repository is no longer maintained
#63Will https://github.com/chainguard-forks/minio hold the fork?
Re: MinIO repository is no longer maintained
#64This is timely news for me - I was just standing up some Loki infrastructure yesterday & following Grafana's own guides on object storage (they recommend minio for non-cloud setups). I wasn't previously experienced with minio & would have completely missed the maintenance status if it wasn't for Checkov nagging me about using latest tags for images & having to go searching for release versions. Sofar I've switched to…
Why do you need non-trivial dependency on the object storage for the database for logs in the first place?
Object storage has advantages over regular block storage if it is managed by cloud, and if it has a proven record on durability, availability and "infinite" storage space at low costs, such as S3 at Amazon or GCS at Google.
Object storage has zero advantages over regular block storage if you run it on yourself:
- It doesn't provide "infinite" storage space - you need to regularly monitor and manually add new physical storage to the object storage.
- It doesn't provide high durability and availability. It has lower availability comparing to a regular locally attached block storage because of the complicated coordination of the object storage state between storage nodes over network. It usually has lower durability than the object storage provided by cloud hosting. If some data is corrupted or lost on the underlying hardware storage, there are low chances it is properly and automatically recovered by DIY object storage.
- It is more expensive because of higher overhead (and, probably, half-baked replication) comparing to locally attached block storage.
- It is slower than locally attached block storage because of much higher network latency compared to the latency when accessing local storage. The latency difference is 1000x - 100ms at object storage vs 0.1ms at local block storage.
- It is much harder to configure, operate and troubleshoot than block storage.
So I'd recommend taking a look at other databases for logs, which do not require object storage for large-scale production setups. For example, VictoriaLogs. It scales to hundreds of terabytes of logs on a single node, and it can scale to petabytes of logs in cluster mode. Both modes are open source and free to use.
Disclaimer: I'm the core developer of VictoriaLogs.
Re: MinIO repository is no longer maintained
#65I've moved my SaaS I'm developing to SeaweedFS, it was rather painless to do it. I should also move away from minio-go SDK to just use the generic AWS one, one day. No hard feelings from my side to MinIO team though.
Re: MinIO repository is no longer maintained
#66I ran a moderately large opensource service and my chronic back pain was cured the day I stopped maintaining the project. Working for free is not fun. Having a paid offering with a free community version is not fun. Ultimately, dealing with people who don't pay for your product is not fun. I learnt this the hard way and I guess the MinIO team learnt this as well.
Completely different situations. None of the MinIO team worked for free. MinIO is a COSS company (commercial open source software). They give a basic version of it away for free hoping that some people, usually at companies, will want to pay for the premium features. MinIO going closed source is a business decision and there is nothing wrong with that. I highly recommend SeaweedFS. I used it in production for a long…
can vouch for SeaweedFS, been using it since the time it was called weedfs and my managers were like are you sure you really want to use that ?
Re: MinIO repository is no longer maintained
#67This is timely news for me - I was just standing up some Loki infrastructure yesterday & following Grafana's own guides on object storage (they recommend minio for non-cloud setups). I wasn't previously experienced with minio & would have completely missed the maintenance status if it wasn't for Checkov nagging me about using latest tags for images & having to go searching for release versions. Sofar I've switched to…
Why do you need non-trivial dependency on the object storage for the database for logs in the first place? Object storage has advantages over regular block storage if it is managed by cloud, and if it has a proven record on durability, availability and "infinite" storage space at low costs, such as S3 at Amazon or GCS at Google. Object storage has zero advantages over regular block storage if you run it on yourself:…
Initially I was just following recommendations blindly - I've never run Loki off-cloud before so my typical approach to learning a system would be to start with defaults & tweak/add/remove components as I learn it. Grafana's docs use object storage everywhere, so it's a lot easier with you're aligned, you can rely more heavily on config parity.
While I try to avoid complexity, idiomatic approaches have their advantages; it's always a trade-off.
That said my first instinct when I saw minio's status was to use filestorage but the rustfs setup has been pretty painless sofar. I might still remove it, we'll see.
Re: MinIO repository is no longer maintained
#68Earlier quoted context omitted.
There's nothing wrong at all with charging for your product. What I do take issue with, however, is convincing everyone that your product is FOSS, waiting until people undertake a lot of work to integrate your product into their infrastructure, and then doing a bait-and-switch. Just be honest since the start that your product will eventually abandon its FOSS licence. Then people can make an informed decision. Or, if…
> then doing a bait-and-switch FOSS is not a moral contract. People working for free owe nothing to no one. You got what's on the tin - the code is as open source once they stop as when they started. The underlying assumption of your message is that you are somehow entitled to their continued labour which is absolutely not the case.
It's a social contract, which for many people is a moral contract.
Re: MinIO repository is no longer maintained
#69They point to AIStor as alternative. Other alternatives: https://github.com/deuxfleurs-org/garage https://github.com/rustfs/rustfs https://github.com/seaweedfs/seaweedfs https://github.com/supabase/storage https://github.com/scality/cloudserver https://github.com/ceph/ceph Among others
Apart from Minio, we tried Garage and Ceph. I think there's definitely a need for something that interfaces using S3 API but is just a simple file system underneath, for local, testing and small scale deployments. Not sure that exists? Of course a lot of stuff is being bolted onto S3 and it's not as simple as it initially claimed to be.
From what I can gather, S3Proxy[1] can do this, but relies on a Java library that's no longer maintained[2], so not really much better.
I too think it would be great with a simple project that can serve S3 from filesystem, for local deployments that doesn't need balls to the walls performance.
Re: MinIO repository is no longer maintained
#70I ran a moderately large opensource service and my chronic back pain was cured the day I stopped maintaining the project. Working for free is not fun. Having a paid offering with a free community version is not fun. Ultimately, dealing with people who don't pay for your product is not fun. I learnt this the hard way and I guess the MinIO team learnt this as well.
Completely different situations. None of the MinIO team worked for free. MinIO is a COSS company (commercial open source software). They give a basic version of it away for free hoping that some people, usually at companies, will want to pay for the premium features. MinIO going closed source is a business decision and there is nothing wrong with that. I highly recommend SeaweedFS. I used it in production for a long…
I can also highly recommend SeaweedFS for development purposes, where you want to test general behaviour when using S3-compatible storage. That's what I mainly used MinIO before, and SeaweedFS, especially with their new `weed mini` command that runs all the services together in one process is a great replacement for local development and CI purposes.