Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Convergence to Kubernetes

medium.com

Post reply on HN
Print view View original

101–110 of 124 posts

Re: Convergence to Kubernetes

#101

Post by erikb »

ajross wrote: post #15

> We have close to 30 teams that run some or all of their workloads on our clusters. Approximately 70% of all HTTP traffic we serve is generated from applications within our Kubernetes clusters. Sounds big. But then per wikipedia: > uSwitch.com [...] allows consumers to compare prices for a range of energy, personal finance, insurance and communications services. And: > On 30 April 2015, the property website firm Zoo…

Interesting to see this post downvoted. In my 1.5 years hands on experience with k8s I found that most of the problems we faced (even in much bigger scenarios) would have been solved quicker with simple Linux administration.

We also found that most people don't just run k8s but run it on VMs, with each k8s node exactly being one VM. So the overhead is even bigger to employ k8s in these scenarios (instead of reducing overhead which is one of the key claims of using containers vs VMs).

Re: Convergence to Kubernetes

#102

Post by de_watcher »

jrs95 wrote: post #26
nh2 wrote: post #16

Earlier quoted context omitted.

> don't routine security updates mean you're constantly rebuilding and redeploying these images? It means you should be constantly rebuilding and redeploying these images. The fewest people I've seen use Docker actually do that. The answer I've heard most commonly so far is "uh hmm ... right, given that I see new CVEs fixed every day, scrolling by in the `apt-get dist-upgrade` I do daily on my desktop, we should prob…

You probably shouldn’t be using Ubuntu base images anyways. The closer you can get to “scratch” the better, and the fewer security related issues you’ll have. For most use cases I think Alpine is a much better base image.

That's more about trust and faith in the maintainers of the distribution (that they won't screw up).

Re: Convergence to Kubernetes

#103

Post by de_watcher »

asfdsfggtfd wrote: post #51
rehemiau wrote: post #41

Earlier quoted context omitted.

Sorry for being off-topic but do you have any links or tips on how to achieve this? My understand was that adding a user to the 'docker' group gives him 'sudo'-equivalent rights.

You can set it up so that the user doesn't have to type sudo docker. But they still effectively have root access via docker. I guess it gives some social pressure not to do superuser things?

It looks like a package manager choice. Dpkg/Apt isn't doing by default what they actually need, so they use Docker instead.

Re: Convergence to Kubernetes

#104

Post by alexandre_m »

de_watcher wrote: post #102
jrs95 wrote: post #26

Earlier quoted context omitted.

You probably shouldn’t be using Ubuntu base images anyways. The closer you can get to “scratch” the better, and the fewer security related issues you’ll have. For most use cases I think Alpine is a much better base image.

That's more about trust and faith in the maintainers of the distribution (that they won't screw up).

It's about having the minimal number of system packages and libraries that your app actually needs.

Alpine is leaner so attack surface is thinner.

Re: Convergence to Kubernetes

#105

Post by alexandre_m »

thawkins wrote: post #35
hinkley wrote: post #17

Earlier quoted context omitted.

> having a single artifact and reproducible builds where a developer can run the docker image locally This is the key thing here. As a team grows it’s easy to get various kinds of learned helplessness. Docker, for its faults, is mostly simple enough that you can expect/insist that the team use it. Which means fewer kinds of surprises at deployment time.

We enforced that by giving our devs linux workstations with no sudo rights (iso27001 requirement), but they have access to the docker daemon, and MUST install everything they need in docker images, its massively increased tools sharing between teams, and forced people to learn docker. We have some devs starting to use kubernetes locally using minikube .

Your devs can pass --privileged to docker run, that gives them root.

Re: Convergence to Kubernetes

#106

Post by chrisgaun »

gnur wrote: post #63
amazingman wrote: post #53

Earlier quoted context omitted.

It seems to me that running Kubernetes as a service on DC/OS might be a decent path to take. Mesosphere seem to be pushing this pretty hard, too. What did you think of DC/OS as a base platform? From an ops perspective, I’m finding a lot to like at least conceptually. Especially the idea of managing one “kind” of cluster that then manages many additional kinds for you. But I have yet to actually use it.

It sounds great, but I've always found the DC/OS marketplace not really flexible enough. I've looked at many services, but in most cases the requirements for running a service from the marketplace were for extremely high throughput situations. I wanted to experiment with arrongoDB but it required a total of 16GB memory (IIRC). Practically, I think kubernetes with helm charts gets will get you at least 90% of what DC/…

Mesos means "middle" and it was designed to handle the declarative infrastructure provisioning for different schedulers... container orchestration schedulers: Aurora (Youtube/Twitter), Titus (Netflix), Marathon and Kubernetes (DC/OS).... analytics schedulers: Spark, Tensorflow, etc.

The idea was always that you should manage this infrastructure just like they are managing containers.

discloser: I am the PM of Kubernetes on DC/OS.

Re: Convergence to Kubernetes

#107

Post by ngehani »

JD557 wrote: post #81
hardwaresofton wrote: post #30

Can anyone who used Docker Swarm/Mesos/Nomad and then switched to Kubernetes comment on anything that was done better by Swarm/Mesos/Nomad? I invested in Kubernetes early and always meant to give the others a try (so I could at least know the differences), but never got a chance to.

We are currently migrating from Mesos/Marathon to Kubernetes. One of the things I miss the most is the auto-refresh in the web interface. I know that's a minor issue, but after getting used to it, it's really weird having to do manual refreshes. Also, the Kubernetes UI feels too clutered, but that's probably just because I'm not used to it

WeaveCloud makes it a whole lot simpler to deploy, monitor, and manager your K8s cluster. http://cloud.weave.works. Give it a shot. Easy to connect to your K8s cluster and get going.

Disclosure: I'm Director of Product for Weaveworks.

Re: Convergence to Kubernetes

#108

Post by tonyarkles »

jacques_chester wrote: post #89
chairmanwow wrote: post #43

Earlier quoted context omitted.

Can you go into detail about what things changed in the rewrites that enabled this reduction in RAM usage?

The JVM is traditionally memory-hungry. There are lines of engineering underway which will change that quite a bit (GraalVM looks particularly promising), but for now seeing a JVM process merrily consuming hundreds of Mb of RAM as the base case is not unusual.

That's exactly it. Plus add in all of the Spring Boot stuff (autodiscovery/autowiring, behaviour that gets added simply because a .jar is on the classpath, etc etc) and you end up with a pretty large footprint for a pretty tiny app. For reference too, the deployment artifacts with Go are ~5MB vs. a 70MB .war file.

We've also just used really simple packages (the built-in net/http server for Go vs. embedded Tomcat, gorilla/mux vs. the Spring class annotations for routing), and I suspect that dramatically cuts down on the footprint too.

Re: Convergence to Kubernetes

#109

Post by chrisgaun »

adamtulinius wrote: post #59
lowbloodsugar wrote: post #42

Earlier quoted context omitted.

Mesos is a two layer system. That is what is better about it. Unfortunately, the most common second layer for reliable service execution is Marathon, and it caused us no end of trouble.

Spot on. The two layer system is both a blessing and a curse. Debugging problems can get really hairy because of the two layer split, and it's just not worth it if one is running Marathon as the only framework. I came to the conclusion that Mesos' biggest strength is giving people the option to write their own framework. It's not difficult, and gives you a lot of power over execution. Marathon itself appears to be ve…

While Mesos always supported a number of container orchestration layers (e.g. Netflix's Titus), you are right in that Marathon used to be the only container orchestrator for DC/OS up until 2017. Mesosphere DC/OS has had a Kubernetes package in GA since March and we've had 6 releases since then to make sure always was newest version of Kubernetes and to add features (Strict Mode).

Disclaimer: I am the PM at Mesosphere.

Re: Convergence to Kubernetes

#110

Post by mmt »

bonesss wrote: post #56
mmt wrote: post #37

Earlier quoted context omitted.

I do certainly wonder if the ever-increasing levels of complexity in the layers of abstraction will backfire in some way soon. It seems the trend has accelerated recently.

Luckily there's an easy fix for that: adding more layers abstraction :D

I'll bet you have an "easy" fix for Social Security, too :)

Joking aside, I certainly understand the benefits of abstraction. As someone always points out in any discussion about ORMs, for example, abstractions are leaky. Whenever one has to learn about the inner workings of what the abstraction is hiding, some of that ease evaporates.

Post reply on HN