Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Who operates at scale without containers?

news.ycombinator.com

Post reply on HN
Print view View original

71–80 of 446 posts

Re: Ask HN: Who operates at scale without containers?

#71

Post by sparsely »

alex_duf wrote: post #45

Hey former Guardian employee here. The Guardian has hundreds of servers running, pretty much all EC2 instances. EC2 images are baked and derived from official images, similarly to the way you bake a docker image. We built tools before docker became the de facto standard, so we could easily keep the EC2 images up to date. We integrated pretty well with AWS so that the basic constructs of autoscaling and load balancer…

> We integrated pretty well with AWS so that the basic constructs of autoscaling and load balancer were well understood by everyone.

This is an underappreciated point I think sometimes. Once you have a team which is familiar with your current, working setup, the benefits of moving away have to be pretty huge for it to be worthwhile.

Re: Ask HN: Who operates at scale without containers?

#73

Post by Negitivefrags »

claytonjy wrote: post #49
Negitivefrags wrote: post #46

We don’t use containers with a pretty large deployment. (1k or so bare metal servers) If you statically link all your binaries then your deployment system can be rsync a directory. The only dependency is that the Linux kernel is new enough. Other than that the packages we deploy will run on literally any setup.

what languages do you use in this way? this sounds a lot easier and more natural for C than python

C++ for the most part.

I don’t use python much but I don’t know why it wouldn’t be possible build it statically and package your entire dependency set into a single directory.

Re: Ask HN: Who operates at scale without containers?

#75

Post by sideway »

smilliken wrote: post #35

My company runs without containers. We process petabytes of data monthly, thousands of CPU cores, hundreds of different types of data pipelines running continously, etc etc. Definitely a distributed system with lots of applications and databases. We use Nix for reproducible builds and deployments. Containers only give reproducible deployments, not builds, so they would be a step down. The reason that's important is t…

Out of curiosity, what industry is your company in?

Re: Ask HN: Who operates at scale without containers?

#76

Post by pipe_connector »

zemo wrote: post #59

when I was at Jackbox we ran the multiplayer servers without containers and handled hundreds of thousands of simultaneous websocket connections (a few thousand per node). The servers were statically compiled Go binaries that took care of their own isolation at the process level and didn't write to disk, they were just running as systemd services. Game servers are inherently stateful, they're more like databases than…

Interesting -- how did you handle redeploys? Given that game servers are stateful (so you'd want to drain servers at their own pace instead of force them down at a specific time), it seems like redeploying a server without machinery to do things like dynamically allocate ports/service discovery for an upstream load balancer would be tricky.

Re: Ask HN: Who operates at scale without containers?

#77

Post by _16k4 »

camtarn wrote: post #41

Don't know if they still use it (I suspect so!) but at least as of 2015 Amazon was using a homebrewed deployment service called Apollo, which could spin up a VM from an internally developed Linux image then populate it with all the software and dependencies needed for a single service. It later inspired AWS CodeDeploy which does the same thing. I remember it being pretty irritating to use, though, since it wasn't par…

I've always thought of Apollo environments as containers before kernel features for containers existed. With enough environment variables and wrapper scripts taking the name of real binaries to populate stuff like LD_LIBRARY_PATH, Apollo makes a private environment that is only _slightly_ contaminated by the host.

Re: Ask HN: Who operates at scale without containers?

#79

Post by abadger9 »

I have a private consulting company which has delivered some pretty sizable footprints (touches most fortune 500 companies via integration with a service) and i prefer deploying without containers. In fact i'll say I hate deploying with containers, which is what i do at my 9-5 and i've lost job opportunities at growth startups because someone was a devout follower of containers and i would rather be honest than using a technology i didn't care for.

Re: Ask HN: Who operates at scale without containers?

#80

Post by motoboi »

freeqaz wrote: post #57
toast0 wrote: post #51

I worked at WhatsApp, prior to moving to Facebook infra, we had some jails for specific things, but mostly ran without containers. Stack looked like: FreeBSD on bare metal servers (host service provided a base image, our shell script would fetch source, apply patches, install a small handful of dependencies, make world, manage system users, etc) OTP/BEAM (Erlang) installed via rsync etc from build machine Application…

That's a very cool deployment method to just use rsync like that! Simple and very composable. And now I feel self-conscious about my pile of AWS and Docker!

Containers are just tar.gz files, you know? The whole layers thing it’s just an optimization.

You can actually very simply run those tar.gz files without docker involved, just cgroups. But then you’ll have to write some daemon scripts to start, stop, restart, etc

Follow this path and soon you’ll have a (worst) custom docker. Try to create a network out of those containers and soon a (worst) SDN network appears.

Try to expand that to optimal node usage and soon a (worst) Kubernetes appears.

My point here is: it’s just software packaged with their dependencies. The rest seems inconsequential, but it’s actually the hard part.

Post reply on HN