Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

You might not need Kubernetes

blog.jessfraz.com

Post reply on HN
Print view View original

181–190 of 319 posts

Re: You might not need Kubernetes

#181

Post by devhead »

TheDong wrote: post #173
m_mueller wrote: post #150

Earlier quoted context omitted.

docker image definitions are idempotent as a matter of principle. creating an idempotent shell script is non trivial IMO - e.g. what return code is returned from package manager XY when something is already installed etc.

Really? What do your dockerfiles look like? Most of the ones I've seen in the wild look something like: FROM debian:jessie RUN apt-get install -y somepkg ... What happens when the "debian:jessie" image changes (as it does weekly on dockerhub)? What about "somepkg" in debian's repositories? The answer is that `docker build` will produce a different image. In fact, very few docker image builds I've seen are idempotent.…

that's why you version pin and vet new updates before you let them in.

Re: You might not need Kubernetes

#182

Post by Jedd »

wpietri wrote: post #87
combatentropy wrote: post #19

Some day I would like a powwow with all you hackers about whether 99% of apps need more than a $5 droplet from Digital Ocean, set up the old-fashioned way, LAMP --- though feel free to switch out the letters: BSD instead of Linux, Nginx instead of Apache, PostgreSQL instead of MySQL, Ruby or Python instead of PHP. I manage dozens of apps for thousands of users. The apps are all on one server, its load average around…

As somebody who has his own colocated server (and has since Bubble 1.0), I definitely agree that the old-fashioned way still works just fine. On the other hand, I've been building a home Kubernetes cluster to check out the new hotness. And although I don't think Kubernetes provides huge benefits to small-scale operators, I would still probably recommend that newbs look at some container orchestration approach instead…

> The problem for me with the old big-server-many-apps approach is the way it becomes hard to manage. 5 years on, I know that I did a bunch of things for a bunch of reasons, but I don't really remember what or why.

I thought this was a solved problem.

I use SaltStack for config management & orchestration on my own machines. (I suggest any config management tool becomes 'worth the effort' once you're managing more than a handful of machines, and/or want to rebuild or spin up new machines with minimal effort and surprises more than once a year.)

Why I do something is described in a comment in the yaml that does the something.

For more nuanced situations, I'll document it in my wiki. (With a yaml comment pointing to same -- I am extremely pessimistic about future-Jedd's ability to remember things.)

If you're running a big-server-many-apps or many-servers-with-their-own-apps, I'd expect the same approach to work equally well.

Though the whole idea of virtual servers & config management (but not necessarily docker or k8s) is that you don't have a bunch of disparate and potentially conflicting apps with potentially conflicting dependencies on a single server.

> But "virtual server" is like "horseless carriage". The term itself indicates that some transition is happening, but that we don't really understand it yet.

That's a challenging assertion to fully unpack. IT's undeniably in a constant state of transition -- and not always thoughtfully directed -- but the problem isn't _'virtual server'_.

The general trend is obviously towards isolation -- but the tooling, performance, scaling, design, and security disparities make the arguments around what level you try to implement your isolation so interesting.

Re: You might not need Kubernetes

#183

Post by wetpaste »

manigandham wrote: post #162
combatentropy wrote: post #19

Some day I would like a powwow with all you hackers about whether 99% of apps need more than a $5 droplet from Digital Ocean, set up the old-fashioned way, LAMP --- though feel free to switch out the letters: BSD instead of Linux, Nginx instead of Apache, PostgreSQL instead of MySQL, Ruby or Python instead of PHP. I manage dozens of apps for thousands of users. The apps are all on one server, its load average around…

It's not just about scaling. That seems to be the only thing people talk about because it sounds sexy but the reality is about operations. Kubernetes makes deployments, rolling upgrades, monitoring, load balancing, logging, restarts, and other ops very easy. It can be as simple as a 1-line command to run a container or several YAML files to run complex applications and even databases. Once you become familiar with th…

Exactly. It solves some of the most important problems that come up of working with microservice based architectures, and establishes mature patterns around the ability for multiple developer teams to update and scale each piece of a distributed application.

Also, what do you do when your digital ocean droplets needs hundreds or thousands of customers on it? Maybe each customer needs it's own database (well in my case they do), configuration, storage requirements, and multi-node requirements. How do you keep track of all that, how do you automatate it and QUICKLY recover in failure scenarios. You need to be able to deal with failure on a node, or if there's a bad actor you need to be able to move them off easily without downtime or affecting other customers, automatically, seamlessly. You need to be able see an overview of your resources across all nodes and where apps are placed and have something decide if the hardware your new container is being added to can handle another JVM or whatever. For cost effectiveness, you want to be able to overcommit resources, so you want containers. You want those to translate to other platforms, AWS, google, azure, on-prem. You have a single declarative language that works anywhere you can deploy a k8s cluster. You need to deal with growth, good patterns for rolling back and updating versions of parts of the stack. You want all of your deployments to be declarative and to be able to tightly control the the options for each one and get back to where you were.

I agree that it doesn't make sense for everything, and it requires fundamental understanding of linux and software before it even makes sense to try shoehorning onto, but it solves real world problems for many people, it's not just a hype thing. I would say docker itself was more of a hype thing than k8s, the maturity and features of k8s and other orchestration systems that came out of the docker model is there for a reason, because it solves all of the real-world problems people couldn't solve with vanilla docker without tons of custom scripting and hacky workarounds. Docker solved the big problem by providing the isolating environments for each app and splitting things out into microservices that way, without having to commit a full statically resourced VM or bare metal per service. K8s solves all of the other problems that came out of that (pods, stateful sets, init containers, jobs, cronjobs, service definitions, deployments, volume claims.)

Re: You might not need Kubernetes

#184

Post by jdhawk »

meritt wrote: post #133
combatentropy wrote: post #19

Some day I would like a powwow with all you hackers about whether 99% of apps need more than a $5 droplet from Digital Ocean, set up the old-fashioned way, LAMP --- though feel free to switch out the letters: BSD instead of Linux, Nginx instead of Apache, PostgreSQL instead of MySQL, Ruby or Python instead of PHP. I manage dozens of apps for thousands of users. The apps are all on one server, its load average around…

As someone who runs a very successful data business on a simple stack (php, cron, redis, mariadb), I definitely agree. We've avoided the latest trends/tools and just keep humming along while outperforming and outdelivering our competitors. We're also revenue-funded so no outside VC pushing us to be flashy, but I will definitely admit it makes hiring difficult. Candidates see our stack as boring and bland, which we ma…

I work in the same environment, and hiring is the only downside I see. Resume Driven Development and "Dev Sexy" has made it difficult to find developers who are willing to come on board, despite the sanity provided by simplicity & comp.

Re: You might not need Kubernetes

#185

Post by nineteen999 »

meritt wrote: post #133
combatentropy wrote: post #19

Some day I would like a powwow with all you hackers about whether 99% of apps need more than a $5 droplet from Digital Ocean, set up the old-fashioned way, LAMP --- though feel free to switch out the letters: BSD instead of Linux, Nginx instead of Apache, PostgreSQL instead of MySQL, Ruby or Python instead of PHP. I manage dozens of apps for thousands of users. The apps are all on one server, its load average around…

As someone who runs a very successful data business on a simple stack (php, cron, redis, mariadb), I definitely agree. We've avoided the latest trends/tools and just keep humming along while outperforming and outdelivering our competitors. We're also revenue-funded so no outside VC pushing us to be flashy, but I will definitely admit it makes hiring difficult. Candidates see our stack as boring and bland, which we ma…

Hear hear. I lead a team that builds and manages critical emergency services infrastructure.

Our stack is pretty boring, but then it has to be running 99.999% of the time. Rather than wasting time chasing the latest flavour-of-the-month tool or framework we invest our time in plugging any kind of gap that could ever bring our service down.

We don't need people who are only looking to pad their resume with the hottest buzzwords, we look for people who want to make critical services run all that time, that rarely fail and when they do, they handle failure gracefully.

The number of devops/agile/SaaS style shops I have seen where the product falls over for several hours at a time is astounding, and it can often be attributed to rushed/inexperienced engineering or unnecessary complexity.

Lucky for them it's usually just the shareholders bottom line that is affected. If the services my team provides don't work, ambulances and fire engines are either not arriving fast enough or at all.

Re: You might not need Kubernetes

#186

Post by devhead »

pathseeker wrote: post #149
dcosson wrote: post #98

Earlier quoted context omitted.

I really think that running a LAMP server for the average beginning developer these days would be just as complicated, maybe more complicated, than running a single deployment on Google Kubernetes Engine. You have to know about package managers and init systems and apache/nginx config files and keep track of security updates for your stack and rotate the logs so the hard drive doesn't fill up. If you already know how…

>Side note - couldn't you make a similar argument about any kind of further abstraction? "Question for all you hackers out there - do you really need HTTP requests with their complicated headers and status codes and keepalive timeouts? I run several apps just sending plain text over TCP sockets and it works fine." No, because the end users already have an HTTP browser. Your analogy doesn't work because switching betw…

maybe the end user is someone sending curl requests?

Re: You might not need Kubernetes

#187

Post by geo_mer »

Kubernetes may be overkill for small projects and it's actually hard to setup for a single-machine cluster, but the idea of container orchestrators (k8s, docker swarm, nomad, etc...) is extremely useful, I understand that some abuse the word "scale", but for me container orchestration is far bigger than just scaling, these features include:

1. rolling updates

2. decoupling configs and secrets from code and mounting/changing config files easily

3. robust and predictable testing/production environments

4. centralized logging

Also microservices's goal isn't really about just "scaling" in my opinion, there are other important advantages even if you have no intention to scale, aspects like modularity, separation of concerns, robustness and lowering the technical debt are still as important whether your app serves 1 or a 10000 users at the same time. Of course you can pull your python app from your repo or even rsync it (just like you can just develop any software without using git or any revision control) and just execute it might work very well, but sooner or later you are going to regret it if you're a business

Re: You might not need Kubernetes

#188

Post by TheDong »

devhead wrote: post #181
TheDong wrote: post #173

Earlier quoted context omitted.

Really? What do your dockerfiles look like? Most of the ones I've seen in the wild look something like: FROM debian:jessie RUN apt-get install -y somepkg ... What happens when the "debian:jessie" image changes (as it does weekly on dockerhub)? What about "somepkg" in debian's repositories? The answer is that `docker build` will produce a different image. In fact, very few docker image builds I've seen are idempotent.…

that's why you version pin and vet new updates before you let them in.

Okay, so walk me through how to do this in a dockerfile?

My first line becomes:

    FROM debian@sha256:14e15b63bf3c26dac4f6e782dbb4c9877fb88d7d5978d202cb64065b1e01a88b
Okay, that's easy.

Now, what about older versions of packages in debian's apt repos that have been deleted? How do I get those?

I run my own apt mirror I guess which I update in lock-step with my dockerfile and thus don't let the Dockerfile reach out to the network.

Is this any different from what you do in a shell script on a server? You use btrfs/zfs/whatever to snapshot the initial version and back it up, you run an apt repository so you can pin package versions, you snapshot before and after updating...

I don't see how a docker image definition makes any of this easy. There's not even a flag to disallow network access during "docker build".

The claim I'm responding to is "docker image definitions are idempotent as a matter of principle".

The large majority of dockerfiles I've seen are not idempotent. Yes, it's possible to make them idempotent, but they do not make it easy.

Re: You might not need Kubernetes

#189

Post by movedx »

SEJeff wrote: post #57
combatentropy wrote: post #19

Some day I would like a powwow with all you hackers about whether 99% of apps need more than a $5 droplet from Digital Ocean, set up the old-fashioned way, LAMP --- though feel free to switch out the letters: BSD instead of Linux, Nginx instead of Apache, PostgreSQL instead of MySQL, Ruby or Python instead of PHP. I manage dozens of apps for thousands of users. The apps are all on one server, its load average around…

FreeBSD, Apache, Python - the FAP stack Ok. I'll see myself out.

I laughed ;-)

Re: You might not need Kubernetes

#190

Post by toredash »

staticassertion wrote: post #139
combatentropy wrote: post #19

Some day I would like a powwow with all you hackers about whether 99% of apps need more than a $5 droplet from Digital Ocean, set up the old-fashioned way, LAMP --- though feel free to switch out the letters: BSD instead of Linux, Nginx instead of Apache, PostgreSQL instead of MySQL, Ruby or Python instead of PHP. I manage dozens of apps for thousands of users. The apps are all on one server, its load average around…

I think that kub and, generally, cloud providers have allowed for more ambitious projects to be generally accessible. My side project is intended to handle > 1 billion events per day, with fairly low latency. That's well over 10k events per second. I doubt I could do this easily on a single box, and I wouldn't really want to try. Why constrain myself that way? Is it worth just doing this the standard LAMP way? More a…

Depends on the box. Per core performance is probably your metric there if the app can utilize multiple cores.
Post reply on HN