Playing with Docker 1.13 and Minio • Hacker News Board

Re: Playing with Docker 1.13 and Minio

#11

Post by unlocksmith » Sun, Jan 22, 2017, 9:58 AM UTC

alrs wrote: post #4

Before anyone gets excited, know that Minio only just started supporting clusters of multiple machines, and that it is severely limited: "As with Minio in stand-alone mode, distributed Minio has a per tenant limit of minimum 4 and maximum 16 drives"[0]. They built a toy object store while blogging aggressively. Openstack Swift or Redhat Ceph are still the only real open-source object store players, AFAIK. [0] http://…

Minio is designed to work in conjunction with Kubernetes and Mesos like external orchestrators. GlusterFS, Ceph and Swift has builtin orchestration. So when you measure Minio's scalability, you should measure Kubernetes instead.

Multi-tenant cloud-native architecture requires each tenant to be intentionally kept small (limited by the failure domain). This way, when you scale from few hundred tenants to millions, your complexity doesn't scale proportionately. Million'th instance of Minio is as simple as its first instance. If you are looking for a global namespace, simply use ngnix like proxy load-balancers in the front. Orchestration tools are well understood at scale unlike Ceph and Swift.

Building monolithic distributed systems has a number of challenges. They do not scale beyond a point. Failure may take down the entire site. Troubleshooting is hard. No CI/CD. Entire system has to be upgraded at once with planned downtime. Security breach exposes the whole system..

-ab Architect (Minio and formerly GlusterFS)

Re: Playing with Docker 1.13 and Minio

#12

Post by durich » Sun, Jan 22, 2017, 1:43 PM UTC

alrs wrote: post #4

Before anyone gets excited, know that Minio only just started supporting clusters of multiple machines, and that it is severely limited: "As with Minio in stand-alone mode, distributed Minio has a per tenant limit of minimum 4 and maximum 16 drives"[0]. They built a toy object store while blogging aggressively. Openstack Swift or Redhat Ceph are still the only real open-source object store players, AFAIK. [0] http://…

Have anyone tried libres3 from Skylable?

Re: Playing with Docker 1.13 and Minio

#13

Post by unlocksmith » Sun, Jan 22, 2017, 10:09 PM UTC

general_ai wrote: post #6

alrs wrote: post #5

Earlier quoted context omitted.

Physically impossible to run a PB-scale Minio cluster: it's limited to 16 hard drives. See my comment above.

Good to know. Looks like more of a PR initiative then. Some of my servers have more drives than that. :-)

Minio shards data across 16 drives. If you have more drives, you would run multiple instances for each set of 16 drives. I would generally recommend against denser storage servers. When it goes down, all the drives go offline with it.