Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Convergence to Kubernetes

medium.com

Post reply on HN
Print view View original

31–40 of 124 posts

Re: Convergence to Kubernetes

#31

Post by lsh »

lkrubner wrote: post #23
ordinaryperson wrote: post #2

Dumb question from someone who doesn't use Kubernetes (or Docker) in production: don't routine security updates mean you're constantly rebuilding and redeploying these images? And if so, how is that more efficient than just using Puppet / Chef / Ansible and a 'real' server?

Docker is a dangerous gamble and you can get more of an automated build system, with less devops effort, from Terraform and Packer. Avoid containers and stick with real servers “baked” by Packer: http://www.smashcompany.com/technology/docker-is-a-dangerous...

that was a good read, thankyou

Re: Convergence to Kubernetes

#32

Post by ajross »

accnumnplus1 wrote: post #20
ajross wrote: post #15

> We have close to 30 teams that run some or all of their workloads on our clusters. Approximately 70% of all HTTP traffic we serve is generated from applications within our Kubernetes clusters. Sounds big. But then per wikipedia: > uSwitch.com [...] allows consumers to compare prices for a range of energy, personal finance, insurance and communications services. And: > On 30 April 2015, the property website firm Zoo…

30 teams' services fit on 2-3 servers? We don't have anywhere near 30 teams, but at my last count end of last year there were ~250 repos, and everyday I get auto-emailed telling me I've been subscribed to a new repo. No way could we have only 2-3 servers.

> 30 teams' services fit on 2-3 servers?

Why would system load scale with the number of engineers? Yeah. 30 "teams" worth of work can totally fit on one system. The sum total of all software engineering ever done before, I dunno, 1979 can fit on one box.

Really, that's my point. People are far too enthused with the "feeling" of working on a "big" project and not being sufficiently reasonable or conservative about the technologies they try to use.

Cluster deployment for an application this size just isn't needed or appropriate. It's cargo cult engineering, because all the cool kids are using k8s and we want to read about cool kids.

Re: Convergence to Kubernetes

#33

Post by MPSimmons »

lkrubner wrote: post #23
ordinaryperson wrote: post #2

Dumb question from someone who doesn't use Kubernetes (or Docker) in production: don't routine security updates mean you're constantly rebuilding and redeploying these images? And if so, how is that more efficient than just using Puppet / Chef / Ansible and a 'real' server?

Docker is a dangerous gamble and you can get more of an automated build system, with less devops effort, from Terraform and Packer. Avoid containers and stick with real servers “baked” by Packer: http://www.smashcompany.com/technology/docker-is-a-dangerous...

There's no reason you can build containers in an automated, routine fashion and use those to run your applications and services on. You don't have to run containers like joeblow/randomservice - start with Alpine from the Alpine maintainers (or CentOS or whatever) and write a custom dockerfile to build your stuff.

Re: Convergence to Kubernetes

#34

Post by RVuRnvbM2e »

lkrubner wrote: post #23
ordinaryperson wrote: post #2

Dumb question from someone who doesn't use Kubernetes (or Docker) in production: don't routine security updates mean you're constantly rebuilding and redeploying these images? And if so, how is that more efficient than just using Puppet / Chef / Ansible and a 'real' server?

Docker is a dangerous gamble and you can get more of an automated build system, with less devops effort, from Terraform and Packer. Avoid containers and stick with real servers “baked” by Packer: http://www.smashcompany.com/technology/docker-is-a-dangerous...

Why would I want the overhead of a hypervisor when I can just use the process isolation features built into the kernel?

Re: your essay

I couldn't quite tell but is it correct to say that you consider Docker a "dangerous gamble" because Docker Inc. may cease to exist in its current form at some point?

Re: Convergence to Kubernetes

#35

Post by thawkins »

hinkley wrote: post #17
keypusher wrote: post #13

Earlier quoted context omitted.

I don't think people primarily choose Docker because it's more efficient(?) to deploy than a real server. The benefits of Docker, to me, are having a single artifact and reproducible builds where a developer can run the docker image locally, it then gets built once on Jenkins, and the same image is deployed to staging, production, etc. It eliminates an entire class of problems related to the operating system, install…

> having a single artifact and reproducible builds where a developer can run the docker image locally This is the key thing here. As a team grows it’s easy to get various kinds of learned helplessness. Docker, for its faults, is mostly simple enough that you can expect/insist that the team use it. Which means fewer kinds of surprises at deployment time.

We enforced that by giving our devs linux workstations with no sudo rights (iso27001 requirement), but they have access to the docker daemon, and MUST install everything they need in docker images, its massively increased tools sharing between teams, and forced people to learn docker. We have some devs starting to use kubernetes locally using minikube .

Re: Convergence to Kubernetes

#36

Post by MPSimmons »

polskibus wrote: post #6

Is there a recommended way to handle database migrations in kubernetes? Is there a best practice or a tool for that?

What do you mean when you say 'database migration'?

In my mind, there could be a few things:

1) Migrating the database from server A to server B where the server is on Kubernetes

A1) Don't do this. Don't run a (traditional) database server in Kubernetes. Sure, you can do this - there are all kinds of volume support for all kinds of things. Everything I've ever heard and read has told me that containers aren't a good for for this type of long-term service that gets refreshed infrequently.

2) Migrating the database from server A to server B where you have to tell all of the clients what the database is

A2) This should probably be done via service discovery or even just by a short TTL on a CNAME in DNS.

3) Something else?

Re: Convergence to Kubernetes

#37

Post by mmt »

101km wrote: post #9

"The result was a system composed of many wavefronts of change: some systems were automated with Puppet, some with Terraform, some used ECS and others used straight EC2. In 2012 we were proud to have an architecture that could evolve so frequently, letting us experiment continually, discovering what worked and doing more of it. In 2017, however, we finally recognised that things had changed. AWS is significantly more…

I do certainly wonder if the ever-increasing levels of complexity in the layers of abstraction will backfire in some way soon.

It seems the trend has accelerated recently.

Re: Convergence to Kubernetes

#38

Post by subway »

MPSimmons wrote: post #36
polskibus wrote: post #6

Is there a recommended way to handle database migrations in kubernetes? Is there a best practice or a tool for that?

What do you mean when you say 'database migration'? In my mind, there could be a few things: 1) Migrating the database from server A to server B where the server is on Kubernetes A1) Don't do this. Don't run a (traditional) database server in Kubernetes. Sure, you can do this - there are all kinds of volume support for all kinds of things. Everything I've ever heard and read has told me that containers aren't a good…

Database migrations typically refer to schema/ddl changes to an application along side deployment of a new version.

Everything I've ever heard and read has told me that containers aren't a good for for this type of long-term service that gets refreshed infrequently.

That was a safe rule of thumb 5 years ago. Since then support for scheduling a persistent volume to be available along side your long running container has become a bog standard and boring feature.

Re: Convergence to Kubernetes

#39

Post by TheDong »

nh2 wrote: post #16
ordinaryperson wrote: post #2

Dumb question from someone who doesn't use Kubernetes (or Docker) in production: don't routine security updates mean you're constantly rebuilding and redeploying these images? And if so, how is that more efficient than just using Puppet / Chef / Ansible and a 'real' server?

> don't routine security updates mean you're constantly rebuilding and redeploying these images? It means you should be constantly rebuilding and redeploying these images. The fewest people I've seen use Docker actually do that. The answer I've heard most commonly so far is "uh hmm ... right, given that I see new CVEs fixed every day, scrolling by in the `apt-get dist-upgrade` I do daily on my desktop, we should prob…

> At the time of writing, every single image is labelled with "This image has vulnerabilities".

The majority of those are not applicable in any way to your average container. I would highly encourage you to not attempt to make points off such bad data. See below for why I think it's bad data.

I'd also like to point out that those CVEs are even more of a pointless thing to make such a point with since almost all of them aren't fixed in upstream ubuntu... which is to say an ubuntu server with 'unattended upgrades' would be just as 'vulnerable' as these docker containers, except moreso because more of them would actually be relevant in such an environment.

Your overall point is valid, but your reference to those 'vulnerabilities' is egregiously misleading.

Let me analyse as a human all the so-called "critical vulnerabilities" listed there for Xenial:

1. glibc 2.23-0ubuntu10 - CVE-2018-6485

This is exploited by C code which intentionally calls posix_memalign or aligned_alloc (instead of malloc) with unusually large arguments. There are very few codebases out there which make use of that in the first place.

If your container is not running untrusted code which links against libc, you have little to fear. This CVE will not impact the average container running some ruby or nodejs application.

Ubuntu also offers no update yet, so it's not actionable.

2. ncurses 6.0+20160213-1ubuntu1 - CVE-2017-10684, CVE-2017-10685

Only affects you if you're piping un-sanitized user input into an ncurses application which then displays it. I doubt there are many, if any, server applications that do this.

Also not actionable.

3. shadow 4.2-3.1ubuntu5.3 - CVE-2017-12424

I'm sure there are plenty of containers out there shelling out to "newusers" with totally unvalidated input. Highly critical I'm sure.

4. cryptsetup 1.6.6-5ubuntu2.1 - CVE-2016-4484

This one only impacts the initrd of luks encrypted setups.... literally impossible to accomplish in a container. Completely garbage listing, no value, cannot possibly impact a docker container.

5. systemd 229-4ubuntu21.2 - CVE-2018-6954

This requires 'systemd-tmpfiles' to run after an attacker has manipulated the filesystem. 'systemd-tmpfiles' is not run in docker containers, and even if it did, since it starts from a fresh root filesystem each time it's a moot point since any changes the attacker makes won't persist to the next time the container "reboots" and tmpfiles runs (assuming the rare case it's run for some reason as part of the container's boot up).

Another non-applicable one.

6. util-linux 2.27.1-6ubuntu3.4 - CVE-2018-7738

This one's a bug in bash-completions for umount, which aren't installed in the container by default. This one's a false positive because the vulnerable code (the bash-completions script) is not present in the docker image.

Re: Convergence to Kubernetes

#40

Post by minieggs »

hardwaresofton wrote: post #30

Can anyone who used Docker Swarm/Mesos/Nomad and then switched to Kubernetes comment on anything that was done better by Swarm/Mesos/Nomad? I invested in Kubernetes early and always meant to give the others a try (so I could at least know the differences), but never got a chance to.

I would love to see a comparison between Docker Swarm and Kubernetes. From talking to peeps I've got that Kubernetes is better for a large number of nodes? I self host all my side projects with Docker Swarm and it's been so good I haven't needed to look into other container management solutions (but I've only got eight nodes).
Post reply on HN