Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Putting GPUs to work with Kubernetes

medium.com

Post reply on HN
Print view View original

11–20 of 54 posts

Re: Putting GPUs to work with Kubernetes

#11

Post by moondev »

Seanny123 wrote: post #7
moondev wrote: post #6

Earlier quoted context omitted.

https://vishh.github.io/docs/concepts/overview/what-is-kuber...

If you'd like to engage with me further, how does a company know it needs Kubernetes? If I'm Soylent and I'm processing a few orders a minute, I'm probably safe with a few redundant monoliths. Do I have to be Uber? What's the middle-ground between Soylent and Uber that would still need this? Is the answer the same as the question "who needs a microservice architecture"?

In my opinion, if you are running containers in prod, you need to be on Kubernetes, regardless of scale.

Kubernetes is so much more than just "planet scale". It encourages patterns and mindsets for efficient software delivery that can really pay dividends.

Here are some of my favorite things:

Cloud agnostic. Your team and business are not at the mercy of pricing, features or availability of a third party. You can run it on everything from a massive cluster on AWS to some cheap mini computers off ebay: https://hackernoon.com/diy-kubernetes-cluster-with-x86-stick... Moving between cloud-providers when they both run Kubernetes is fairly trivial. You can also run on multiple clouds at the same time. Kubernetes abstracts the infrastructure away. It's also really easy to run a single node cluster on your own machine for local development. Try doing that with AWS services in a reliable way.

Immutable infrastructure:

The fact that containers don't hold state FORCES you to develop your applications in a 12-factor pattern. Deploy images by tag forces you to create a pipeline that automates their builds. It also allows you to effortlessly roll-back. It's not an afterthought or something you need to glue together.

High availability:

Just define how many replicas of your service you want and k8s does the rest. If they crash so what. Not only will they be restarted automatically but they will automatically be distributed across your fleet for you. Node goes down? Who cares. It's self-healing.

Service discovery:

Just put a k8s service in front of your application replicas and everything is automatic. Nothing to install, simply refer to the stable dns service name and everything will be routed. Software agnostic.

Config Management:

Very easy to inject secrets and configs as env vars or mounted into the pod. No third party library or framework needed to leverage it.

Dev - Stage - Prod envs made easy:

The same container image can move through each env effortlessly and you can be sure there is no "artifact rot"

Extensible and open

You can run different container runtimes such as rkt or different pod networks and persistent storage options. There is not a single company trying to steer it in some way. Also recently with helm charts it's becoming very easy to "apt-get install" whatever you want on your cluster. Very powerful and portable.

It does take some time getting ramped up but once it clicks there is no turning back.

Re: Putting GPUs to work with Kubernetes

#13

Post by tokenizerrr »

samnco wrote: post #9
Seanny123 wrote: post #7

Earlier quoted context omitted.

If you'd like to engage with me further, how does a company know it needs Kubernetes? If I'm Soylent and I'm processing a few orders a minute, I'm probably safe with a few redundant monoliths. Do I have to be Uber? What's the middle-ground between Soylent and Uber that would still need this? Is the answer the same as the question "who needs a microservice architecture"?

There are a few killer features that you would benefit at any size and that I really love * self healing: when you create a deployment/replica set. it will be maintained at all cost, so if the app has a memory leak or anything goes wrong, it will be contained and kept up and running * Rolling update: even when you run 5 frontends, it is a pain to use capistrano or other tools to just update at git repo. it is literal…

When running a kubernetes cluster on your own hardware what do you use for storage?

Re: Putting GPUs to work with Kubernetes

#14

Post by eicnix »

tokenizerrr wrote: post #13
samnco wrote: post #9

Earlier quoted context omitted.

There are a few killer features that you would benefit at any size and that I really love * self healing: when you create a deployment/replica set. it will be maintained at all cost, so if the app has a memory leak or anything goes wrong, it will be contained and kept up and running * Rolling update: even when you run 5 frontends, it is a pain to use capistrano or other tools to just update at git repo. it is literal…

When running a kubernetes cluster on your own hardware what do you use for storage?

When you are running on your own hardware you usually have multiple options for storage:

- Use the local node storage although this is very simple but can get complicated on more complex installations

- Connect to your existing storage solution using ISCSI or NFS

- Running your own distributed storage solution on top of Kubernetes for Kubernetes. e.g. https://github.com/rook/rook

Re: Putting GPUs to work with Kubernetes

#15

Post by samnco »

tokenizerrr wrote: post #13
samnco wrote: post #9

Earlier quoted context omitted.

There are a few killer features that you would benefit at any size and that I really love * self healing: when you create a deployment/replica set. it will be maintained at all cost, so if the app has a memory leak or anything goes wrong, it will be contained and kept up and running * Rolling update: even when you run 5 frontends, it is a pain to use capistrano or other tools to just update at git repo. it is literal…

When running a kubernetes cluster on your own hardware what do you use for storage?

You have several options:

* Run Ceph in separate nodes and connect it to the cluster. With Juju, you can do that from the bundle, as Ceph is also a supported workloads. This gives you scale for storage

* Run Ceph within the cluster with a Helm chart. We see that for openstack-helm for example. Also gives you scale, but the lack of device discovery makes it less interesting in my opinion

* Run an NFS server, plain easy but not very scalable.

* Use hostpath, which is the default but doesn't get you scale.

Re: Putting GPUs to work with Kubernetes

#16

Post by aquadrop »

eicnix wrote: post #14
tokenizerrr wrote: post #13

Earlier quoted context omitted.

When running a kubernetes cluster on your own hardware what do you use for storage?

When you are running on your own hardware you usually have multiple options for storage: - Use the local node storage although this is very simple but can get complicated on more complex installations - Connect to your existing storage solution using ISCSI or NFS - Running your own distributed storage solution on top of Kubernetes for Kubernetes. e.g. https://github.com/rook/rook

So, how well is Kubernetes suited for working with local hardware? As I understand it's mainly supposed to work with some external abstracted storage, like AWS EBS, cephs, NFS etc But it's much slower that local SSD and for some small local installation maybe be not optimal. Like, running some not large non-critical service which requires database, several workers, some monitoring etc, overall 3-4 local servers. Is Kubernetes a good fit for this or it's only supposed to work with hive-like stateless workers connected to external storage over network?

Re: Putting GPUs to work with Kubernetes

#17

Post by samnco »

aquadrop wrote: post #16
eicnix wrote: post #14

Earlier quoted context omitted.

When you are running on your own hardware you usually have multiple options for storage: - Use the local node storage although this is very simple but can get complicated on more complex installations - Connect to your existing storage solution using ISCSI or NFS - Running your own distributed storage solution on top of Kubernetes for Kubernetes. e.g. https://github.com/rook/rook

So, how well is Kubernetes suited for working with local hardware? As I understand it's mainly supposed to work with some external abstracted storage, like AWS EBS, cephs, NFS etc But it's much slower that local SSD and for some small local installation maybe be not optimal. Like, running some not large non-critical service which requires database, several workers, some monitoring etc, overall 3-4 local servers. Is K…

You have several options for this. If it is non HA, then you can pin a RC to a specific node, and use hostpath storage. if the container fails, it will always respawn on the same node, maximizing uptime and also having max capacity from your local SSD. Alternatively, you can also run rook, which is backed by Ceph, and use affinity to make sure that your pods are very close to storage, and gain back some of the speed.

In general and in my opinion, it is always better to run with k8s since you have, for the stateless pieces, cluster awareness. So there is never a downside to it, especially as the control plane is very lightweight for small clusters, and you can colocate many parts.

Re: Putting GPUs to work with Kubernetes

#18

Post by eicnix »

aquadrop wrote: post #16
eicnix wrote: post #14

Earlier quoted context omitted.

When you are running on your own hardware you usually have multiple options for storage: - Use the local node storage although this is very simple but can get complicated on more complex installations - Connect to your existing storage solution using ISCSI or NFS - Running your own distributed storage solution on top of Kubernetes for Kubernetes. e.g. https://github.com/rook/rook

So, how well is Kubernetes suited for working with local hardware? As I understand it's mainly supposed to work with some external abstracted storage, like AWS EBS, cephs, NFS etc But it's much slower that local SSD and for some small local installation maybe be not optimal. Like, running some not large non-critical service which requires database, several workers, some monitoring etc, overall 3-4 local servers. Is K…

You can run distributed stateful workloads on kubernetes with local storage and disable rescheduling when the node goes down. This means that you need to manually migrate/restart workloads if a node goes down. There is work being done on improving handling local node storage https://github.com/kubernetes/community/pull/306 but it doesn't scale as well as network storage.

If you have an existing SAN solution you can connect to it via fiber channel over iSCSI.

Re: Putting GPUs to work with Kubernetes

#19

Post by aquadrop »

samnco wrote: post #17
aquadrop wrote: post #16

Earlier quoted context omitted.

So, how well is Kubernetes suited for working with local hardware? As I understand it's mainly supposed to work with some external abstracted storage, like AWS EBS, cephs, NFS etc But it's much slower that local SSD and for some small local installation maybe be not optimal. Like, running some not large non-critical service which requires database, several workers, some monitoring etc, overall 3-4 local servers. Is K…

You have several options for this. If it is non HA, then you can pin a RC to a specific node, and use hostpath storage. if the container fails, it will always respawn on the same node, maximizing uptime and also having max capacity from your local SSD. Alternatively, you can also run rook, which is backed by Ceph, and use affinity to make sure that your pods are very close to storage, and gain back some of the speed.…

Thanks for the answer. How easy will it be to transfer this cluster to another set of servers (with data copy)? Like, stop the service for several minutes, push the button "Transfer" and start service on new servers after that. As I understand you'll need rook for something like that?

Re: Putting GPUs to work with Kubernetes

#20

Post by tokenizerrr »

samnco wrote: post #15
tokenizerrr wrote: post #13

Earlier quoted context omitted.

When running a kubernetes cluster on your own hardware what do you use for storage?

You have several options: * Run Ceph in separate nodes and connect it to the cluster. With Juju, you can do that from the bundle, as Ceph is also a supported workloads. This gives you scale for storage * Run Ceph within the cluster with a Helm chart. We see that for openstack-helm for example. Also gives you scale, but the lack of device discovery makes it less interesting in my opinion * Run an NFS server, plain eas…

So Ceph is the preferred storage provider? I've noticed there is a huge list, including GlusterFS. Do you have experience with any of the other ones?
Post reply on HN