Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Who operates at scale without containers?

news.ycombinator.com

Post reply on HN
Print view View original

141–150 of 446 posts

Re: Ask HN: Who operates at scale without containers?

#141

Post by hinkley »

cj wrote: post #95
Thaxll wrote: post #84

Earlier quoted context omitted.

The thing is, you spent probably a lot of time on things that were granted elsewhere and while the rest of the world is improving on those tech you keep your home grown solution that is harder and harder to maintain. Plus the knowledge that is not transferable. > Containers only give reproducible deployments, not builds, so they would be a step down. This is not true, if you use a docker image A with specific version…

> the rest of the world is improving on those tech Does that matter if the current stack works just fine? Imagine someone who built a calculator app with jQuery javascript 10 years ago, and all it does is add and subtract numbers. You could spend time porting it to Ember, and then migrating to Angular, and then porting it to React, and then porting it to React with SSR and hooks. If the calculator app worked with 15…

The people who are invested in their home-grown solutions are usually pretty bad at doing tech support for them. They enjoy writing code, and it doesn't take long before you've created enough surface area that you couldn't possibly keep up with requests even if you wanted to.

Which they don't, because nothing has convinced me of the crappiness of some of my code more thoroughly than watching other people try and fail to use it. They are a mirror and people don't always like what they see in it. That guy who keeps talking trash about how stupid everyone else is for not understanding his beautiful code is deflecting. If they're right then my code does not have value/isn't smart which means I don't have value/am not smart, so clearly the problem is that I'm surrounded by assholes.

I don't think we would be using as much 3rd party software without StackOverflow, and almost nobody creates a StackOverflow for internal tools. Nobody writes books for internal tools. Nobody writes competitors for internal tools which either prove or disprove the utility of the original tool. All of those options are how some people learn, and in some cases how they debug. You're cutting off your own nose when you reinvent something and don't invest the time to do all of these other things. If I lose someone important or get a big contract I didn't think I could win, I can't go out and hire anyone with 3 years experience in your internal tool, trading money for time. I have to hire noobs and sweat it out while we find out if they will ever learn the internal tools or not.

The older I get the more I see NIH people as working a protection racket, because it amplifies the 'value' of tenure at the company. The externalities created really only affect new team members, delaying the date when you can no longer dismiss their ideas out of hand. You have in effect created an oligarchy, whereas most of the rest of us prefer a representative republic (actual democracy is too many meetings).

Re: Ask HN: Who operates at scale without containers?

#142

Post by tailspin2019 »

gen220 wrote: post #97
tailspin2019 wrote: post #17

Earlier quoted context omitted.

I’m in the process of moving to exactly this approach. I’ve been trying to pick the right Linux distro to base my images on. Ubuntu Server is the low effort route but a bit big to redeploy constantly. I’ve also been looking at the possibility of using Alpine Linux which feels like a better fit but a bit more tweaking needed for compatibility across cloud providers. Unikernels are also interesting but I think that mig…

Google has used debian as their base. Netflix uses a BSD flavor (I forget which) as their CDN cache. FB used CentOS, not sure what they use today since CentOS is EOL'd. Debian (and, formerly, CentOS) is a good standard: it occupies a sweet spot between ubuntu server and alpine, in the sense that it's batteries-included and very well-supported (apt/yum), but not particularly bloated. I use debian for all my personal s…

Useful insight, thanks. What attracts you to BSD over Debian if you were to go that route at some point?

Re: Ask HN: Who operates at scale without containers?

#143

Post by jmt_ »

rootusrootus wrote: post #69
smilliken wrote: post #35

My company runs without containers. We process petabytes of data monthly, thousands of CPU cores, hundreds of different types of data pipelines running continously, etc etc. Definitely a distributed system with lots of applications and databases. We use Nix for reproducible builds and deployments. Containers only give reproducible deployments, not builds, so they would be a step down. The reason that's important is t…

> But I'd so often dive in to understand the details and be disillusioned about how much complexity there is for relatively little benefit. I feel similarly about many cloud services in e.g. AWS. I get where it's sometimes handy, but the management overhead can get pretty insane. I used to hear people say "just do X in AWS and ta-da you're done!" only to find out that in many cases it isn't actually a net improvement…

Totally agree. Something I find myself thinking/saying more and more is "don't underestimate what you can accomplish with a $5 VPS". Maybe a little hyperbolic, but computers are really fast these days.

Elastic resources are pretty magical when you need them, but the majority of software projects just don't end up requiring as many resources as devs/management often seem to assume. Granted, migrating to something like AWS, if you do end up needing it, is a pain, but so is adding unnecessary layers of AWS complexity for your CRUD app/API backend.

Also wonder how many devs came up on AWS and prefer it for familiarity and not having to know and worry too much about aspects of deployment outside of the code itself (i.e not having to know too much about managing postgres, nginx, etc).

Re: Ask HN: Who operates at scale without containers?

#144

Post by [deleted] »

q3k wrote: post #10

Depends what you mean by 'container runtime' or 'container orchestration tool'... For example, Google's Borg absolutely uses Linux namespacing for its workloads, and these workloads get scheduled automatically on arbitrary nodes, but this doesn't feel at all like Docker/OCI containers (ie., no whole-filesystem image, no private IP address to bind to, no UID 0, no control over passwd...). Instead, it feels much closer…

[deleted]

Re: Ask HN: Who operates at scale without containers?

#146

Post by kuon »

yabones wrote: post #82
kuon wrote: post #55

We use ansible on bare metal (no VM) to manage about 200 servers in our basement. We use PXE booting to manage the images. We use a customized arch linux image and we have a few scripts to select what feature we'd like. It's "old school" but it's been working fine for nearly 20 years (we used plain scripts before ansible, so we always used the "agentless" approach). Our networking stack uses OpenBSD.

That sounds really interesting. If you have a write-up about how it's built, the decisions that went into it, and the problems you've had to solve, I would absolutely read it!

We have no public writing, but that could be interesting.

Re: Ask HN: Who operates at scale without containers?

#147

Post by tikkabhuna »

The finance company I work at has historically been copying jars and booting them up with some scripts. Servers all bare metal and some tuned for performance. We have a lot of pet servers as different users needed different tools and asked a sysadmin to install it.

We’re now heavily moving towards containers and the primary motivator is choice of languages and standardisation. Being bare metal is fine when you use a single language, but you’ll find you shoehorn other languages with the same process. Interpreted languages (Node/Python) are a nightmare and you’ll have to find a pattern for running multiple versions on the same host.

Containers is just a real nice deployment unit and is well supported by other tools.

If you are really keen on this path, do consider up front how you will handle version upgrades of the runtime or dependencies.

Re: Ask HN: Who operates at scale without containers?

#148

Post by tra3 »

tharne wrote: post #89
smilliken wrote: post #35

My company runs without containers. We process petabytes of data monthly, thousands of CPU cores, hundreds of different types of data pipelines running continously, etc etc. Definitely a distributed system with lots of applications and databases. We use Nix for reproducible builds and deployments. Containers only give reproducible deployments, not builds, so they would be a step down. The reason that's important is t…

> What works for us is to do the simplest thing that works, then iterate. The older I get, the more often I'm reminded that this un-sexy approach is really the best way to go. When I was younger, I always thought the old guys pushing boring solutions just didn't want to learn new things. Now I'm starting to realize that after several decades of experience, they simply got burned enough times to learn a thing or two h…

Choose boring technology [0].

For me, the choice is a trade off between the journey and the destination.

Destination is the final objective of your project, business value or whatever.

The journey is tinkering with "stuff".

Depending on the project there's value and enjoyment in both.

[0]: https://mcfunley.com/choose-boring-technology

Re: Ask HN: Who operates at scale without containers?

#149

Post by basisword »

usaphp wrote: post #126
tharne wrote: post #89

Earlier quoted context omitted.

> What works for us is to do the simplest thing that works, then iterate. The older I get, the more often I'm reminded that this un-sexy approach is really the best way to go. When I was younger, I always thought the old guys pushing boring solutions just didn't want to learn new things. Now I'm starting to realize that after several decades of experience, they simply got burned enough times to learn a thing or two h…

Or it could be that you just became old and don't want to learn new things anymore ))

Or their experience helps them have the wisdom to determine which new things are worth learning and which are a waste of time.

Re: Ask HN: Who operates at scale without containers?

#150

Post by rapsey »

smilliken wrote: post #35

My company runs without containers. We process petabytes of data monthly, thousands of CPU cores, hundreds of different types of data pipelines running continously, etc etc. Definitely a distributed system with lots of applications and databases. We use Nix for reproducible builds and deployments. Containers only give reproducible deployments, not builds, so they would be a step down. The reason that's important is t…

I've seen comments on here from people with large Nix deployments with like hundreds of thousands of Nix script code and saying it is a complete nightmare to manage.
Post reply on HN