Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Who operates at scale without containers?

news.ycombinator.com

Post reply on HN
Print view View original

251–260 of 446 posts

Re: Ask HN: Who operates at scale without containers?

#251

Post by Art9681 »

anonymousDan wrote: post #139
jandrewrogers wrote: post #131

Earlier quoted context omitted.

Sure, Kubernetes can run a database, but not efficiently. Companies with intensive data infrastructure frequently operate at scale without containers, either VMs or bare metal. The larger the data volume, the less likely they are to use containers because efficiency is more important. It is also simpler to manage this kind of thing outside containers, frankly, since you are running a single process per server. People…

Can you point to any resources on the perf implications of docker for databases?

OP is likely making assumptions on obsolete knowledge. Tech moves really fast which means the knowledge we gained last week might no longer be relevant. Here is an article showing it is perfectly acceptable to deploy Postgres in Kubernetes:

https://www.redhat.com/en/resources/crunchy-data-and-openshi...

The great majority of Kubernetes experts I have met use a managed service and have not been exposed to the internals or the control plane back end. An abstracted version of Kubernetes that works well for most use cases but makes it more difficult to solve some of the problems listed in this thread.

Most folks dont really know Kubernetes. They know how to deploy X application using X abstraction (Helm, Operators, CD tooling) to some worker node in a public cloud using public sources and thats basically it. It's no wonder they cannot solve some of the common problems because they didnt really deploy the apps. They filled out a form and pushed a button.

Re: Ask HN: Who operates at scale without containers?

#252

Post by tonyarkles »

nunez wrote: post #187
smilliken wrote: post #134

Earlier quoted context omitted.

You already got a few good answers, but I'll echo them: you can do reproducible builds in containers, and nothing's stopping you from using nix inside containers. But you're at the mercy of all the different package managers that people will end up using (apt, npm, pip, make, curl, etc). So your system is only as good as the worst one. I inherited a dozen or so docker containers a while back that I tried to maintain.…

I don't know Nix and can't comment on that, but in my experience, when I've inherited containers that couldn't build, this was usually due to its image orphaned from their parent Dockerfiles (i.e. someone wrote a Dockerfile, pushed an image from said Dockerfile, but never committed the Dockerfile anywhere, so now the image is orphaned and unreproducable) or due to the container being mutated after being brought up wi…

So as a quick example from my past experiences, using an Ubuntu base image is fraught. If you don't pin to a specific version (e.g. pinning to ubuntu:20.04 instead of ubuntu:focal-20220316), then you're already playing with a build that isn't reproducible (since the image you get from the ubuntu:20.04 tag is going to change). If you do pin, you have a different problem: your apt database, 6 months from now, will be out of date and lots of the packages in it will no longer exist as that specific version. The solution is "easy": run an "apt update" early on in your Dockerfile... except that goes out onto the Internet and again becomes non-deterministic.

To make it much more reproducible, you need to do it in two stages: first, pinning to a specific upstream version, installing all of the packages you want, and then tagging the output image. Then you pin to to that tag and use that to prep your app. That's... probably... going to be pretty repeatable. Only downside is that if there is, say, a security update released by Ubuntu that's relevant, you've now got to rebuild both your custom base and your app, and hope that everything still works.

Re: Ask HN: Who operates at scale without containers?

#253

Post by goodpoint »

tomc1985 wrote: post #248
rattlesnakedave wrote: post #229

Earlier quoted context omitted.

Or is cope for a loss of neuroplasticity? Has to be evaluated case by case. “Experience” doesn’t count for much when evaluating new technical tooling. Landscape shifts far too much.

The landscape doesn't need to shift nearly as often as it does. But we are an industry obsessed with hiring magpie developers as cheaply as we can, and those magpie developers demand cool merit badges to put on their resumes, and here we are. "Everything old is new again" does not BEGIN to do tech justice. Compared to nearly every other profession on the planet, tech knowledge churn is like 10x as fast.

> tech knowledge churn is like 10x as fast.

Spot on! Moreover, every time the wheel is unnecessarily reinvented previous lessons are lost.

What is even worse is that people really want to reinvent the wheel and get defensive if you point that out.

Comments like "Or is cope for a loss of neuroplasticity" are a good example.

Re: Ask HN: Who operates at scale without containers?

#254

Post by zokier »

dolni wrote: post #6
Nextgrid wrote: post #2

I've been at a company where they weren't (yet) using containers nor K8S. The build process would just create VM images with the required binaries in there and then deploy that to an autoscaling group. It worked well and if you only ever intend to run a single service per machine then is the right solution.

At my workplace we use Docker to run services, but there is no container orchestration like Kubernetes. An AMI bakes in some provisioning logic and the container image. Autoscaling does the rest. Even without orchestration, I argue containers are useful. They abstract the operating system from the application and allow you to manage each independently. Much more easily than you'd be able to otherwise, anyway. Plus yo…

> An AMI bakes in some provisioning logic and the container image

That does sound bit weird setup to me; with something like Packer building an AMI is almost as easy as building a Docker image with Dockerfiles so the benefits of using Docker seem quite slim?

Re: Ask HN: Who operates at scale without containers?

#255

Post by Art9681 »

z3t4 wrote: post #228
motoboi wrote: post #80

Earlier quoted context omitted.

Containers are just tar.gz files, you know? The whole layers thing it’s just an optimization. You can actually very simply run those tar.gz files without docker involved, just cgroups. But then you’ll have to write some daemon scripts to start, stop, restart, etc Follow this path and soon you’ll have a (worst) custom docker. Try to create a network out of those containers and soon a (worst) SDN network appears. Try t…

You can do a lot with just SystemD today. Would not get surprised if SystemD would eat up Docker too.

I'm personally interested in Unikernels. That tech seems to be a good contender for replacing K8s. Every app an appliance running in a hypervisor.

Re: Ask HN: Who operates at scale without containers?

#256

Post by pipe_connector »

zemo wrote: post #217
pipe_connector wrote: post #196

Earlier quoted context omitted.

How do you handle deploying a new version of nginx without forcibly closing connections?

you don't. That requires new nodes, so you have to drain nodes and replace them with new nodes that have the upgrade installed before adding them to the pool that hosts games. That process sucked but how often are you upgrading nginx?

Very often if your load balancer is custom. For example, we have an edge service that fulfils the role you have nginx for, but it handles both websockets and raw tcp traffic. Our edge service is the gateway for authentication and authorization -- from that service we can connect users to chat rooms, matchmaking, or actual game instances. We could get away with just nginx + room ids and manual upgrades to the pool of game/matchmaking/chat services for internal traffic possibly, but we ship updates to our edge service all the time and that process needs to be painless, so I was curious how others have done this.

We're currently using systemd sockets with Accept=no, then multiple edge services can accept() from the same socket that's always open and bound to a known port. Once a new service has started, we can signal the old service to shutdown for however long it needs by no longer listening on the socket and letting connections drain naturally. We're thinking about changing to dynamically allocated ports/sockets which is pretty natural in the container orchestration world.

Re: Ask HN: Who operates at scale without containers?

#257

Post by goodpoint »

spicybright wrote: post #167
repomies69 wrote: post #160

Earlier quoted context omitted.

Sometimes yes. But it is impossible to know for sure. New things still get adopted, some of those new things actually stick and provide value

And at the very least, being old lets you be able to evaluate these new things more accurately.

It obviously does, but experience is not the only way: you can always read what people wrote and did in the past. It only takes some humility.

Re: Ask HN: Who operates at scale without containers?

#258

Post by Art9681 »

ryanjkirk wrote: post #207
freebuju wrote: post #174

Earlier quoted context omitted.

All the so-called "docker killers" are essentially unfinished products. They don't compare 1:1 to docker in feature set and even if they run as rootless, they still are vulnerable to namespace exploits in the Linux kernel. Though docker runs as root, it's still well protected out-of-the-box for the average user and is a very mature technology.

Are you from 2018? Everyone running OpenShift is using CRI-O and that footprint is not small. We made the switch in our EKS and vanilla k8s clusters in 2021. Docker has now even made their API OCI-compliant in order to not be left behind. And the point is that most people don't want a docker feature-for-feature running in prod. The attack surface is simply too large. I don't need an API server running as root on all…

Agreed. Tons of obsolete assumptions in this thread. We have been using Podman / OpenShift in production and never ran into a use case where Docker was needed.

Re: Ask HN: Who operates at scale without containers?

#259

Post by tomc1985 »

goodpoint wrote: post #253
tomc1985 wrote: post #248

Earlier quoted context omitted.

The landscape doesn't need to shift nearly as often as it does. But we are an industry obsessed with hiring magpie developers as cheaply as we can, and those magpie developers demand cool merit badges to put on their resumes, and here we are. "Everything old is new again" does not BEGIN to do tech justice. Compared to nearly every other profession on the planet, tech knowledge churn is like 10x as fast.

> tech knowledge churn is like 10x as fast. Spot on! Moreover, every time the wheel is unnecessarily reinvented previous lessons are lost. What is even worse is that people really want to reinvent the wheel and get defensive if you point that out. Comments like "Or is cope for a loss of neuroplasticity" are a good example.

It's outright naked ageism.

I have been studying a different trade, completely unrelated to software engineering or tech, and by far the weirdest thing is reading books or watching seminars from 20 or 30 years ago that are still relevant. How much of technology writing has that honor? Very little.

Like I have been obsessed with tech and computers my whole life, I studied computer stuff when all the other kids were outside playing football or chasing girls or whatever, I self-taught to a very high level of expertise and that knowledge has just been getting systematically ripped away. Which is fine, to an extent, gotta embrace change and yada yada yada. But I no longer have the patience to grind to expertise-level on something over a period of years only to find out the industry has moved on to something completely different to do the exact same thing! All because some starry-eyed shit somewhere in FAANG doesn't like something!

Re: Ask HN: Who operates at scale without containers?

#260

Post by jimbob45 »

wanderr wrote: post #28

Grooveshark didn't use any of that. We were very careful about avoiding dependencies where possible and keeping our backend code clean and performant. We supported about 45M MAU at our biggest, with only a handful of physical servers. I'm not aware of any blog posts we made detailing any of this, though. And if you're not familiar with the saga, Grooveshark went under for legal, not technical reasons. The backend API…

I miss Grooveshark to this day. Thanks for building such an excellent product!
Post reply on HN