Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Who operates at scale without containers?

news.ycombinator.com

Post reply on HN
Print view View original

401–410 of 446 posts

Re: Ask HN: Who operates at scale without containers?

#401

Post by tailspin2019 »

mst wrote: post #192
tailspin2019 wrote: post #142

Earlier quoted context omitted.

Useful insight, thanks. What attracts you to BSD over Debian if you were to go that route at some point?

Having an operating system where the kernel and core userland are developed together is a qualitatively different experience to one that's been stitched together from external projects. I'm not really sure how to describe the difference in experience but the "this is an integrated whole" feel is really quite pleasant as a sysadmin and as a user. Note that my personal infrastructure is a mixture of Debian and FreeBSD…

You've definitely piqued my interest on BSD!

Re: Ask HN: Who operates at scale without containers?

#402

Post by tomc1985 »

iamstupidsimple wrote: post #397
tomc1985 wrote: post #259

Earlier quoted context omitted.

It's outright naked ageism. I have been studying a different trade, completely unrelated to software engineering or tech, and by far the weirdest thing is reading books or watching seminars from 20 or 30 years ago that are still relevant . How much of technology writing has that honor? Very little. Like I have been obsessed with tech and computers my whole life, I studied computer stuff when all the other kids were o…

The vast majority of computer science lectures from 20 years ago are still relevant. A ton has changed in terms of frameworks but the fundamentals are mostly the same.

It is, but it isn't. Besides, from my vantage point almost all the work is herding frameworks of some kind. Maybe I need to get out of web tech and into something else.

Re: Ask HN: Who operates at scale without containers?

#403

Post by gnufx »

freemint wrote: post #375
gnufx wrote: post #290

Earlier quoted context omitted.

I'd hope the filesystem isn't provided from the login node, but otherwise, yes. That said, they may still run jobs with unprivileged -- I hope -- container images. There are some that seem to feel the need to make life more complicated, but then you have N problems compared with stateless nodes off a networked root running basically native binaries.

I never seen a HPC center where use of containers was encouraged. It is just running binaries from user directories on disk under different users. Attempts to introduce containers in a way which doesn't stand in the way are treated as revolutionary. Such as the Fuzzball scheduler (I found no good written source on a quick skim so have a talk) https://youtu.be/Pbmxq3dg35E

I don't know about encouraged, but HPC-oriented container systems are certainly demanded, particularly Singularity (which I'd avoid). There was Shifter originally and now also at least Charliecloud and Enroot. The normal resource manager could do the namespacing itself, which is what you really want, but it's often possible just to run with an unpacked distribution and suitable environment variables in lieu of a different root.

Re: Ask HN: Who operates at scale without containers?

#404

Post by javajosh »

DeathArrow wrote: post #373
javajosh wrote: post #127

Earlier quoted context omitted.

What you're asking for is an essay on comparative devops architectures, with a focus on k8s alternatives. I think what you'll find is a lot of ad hoc persistent systems that tend to drift over time in unpredictable ways, and take on the feel of a public lobby if you're being generous, a public restroom if you're not. So what you're asking is really a sample of these ad hoc approaches. What I think you'll find are a f…

>I can't imagine why you'd need more than 16 cores, 128G of RAM and 10T of storage to prove out an ordinary SaaS. That is a ferocious amount of resources.) In my case because I was working for a checkout solution for a Cash & Carry chain, with 1000 stores in 25 countries, each with tens of tills, self service points, queue busting tills who also had to serve the web shop. And that solution had to work also when there…

Your project is significantly more complex than an 'ordinary SaaS'. My own canonical example of a SaaS would be something like WuFoo, or Squarespace. These are characterized as pure webapp component producing technologies, from soup to nuts, with through-the-web tooling for everything from specializing your components, to managing your payment details. The system is "closed" in the sense that it only produces and consumes web components for all operations. There is no real-world device that is critical to operation - so you are free to run services like these on any device, or collection of devices, as long as its users can find 'a useful stateful service' at 'name'. For your project, this is only the first foundational requirement.

Re: Ask HN: Who operates at scale without containers?

#405

Post by treis »

spion wrote: post #316
treis wrote: post #247

Earlier quoted context omitted.

>Getting repeatable docker images But there's no real need for repeatable build docker images. You copy the image and run it where ever you need to. The entire point of Docker is to not have to repeat the build.

Until there is a critical vulnerability in one of the components present on that image (system packages or application packages).

At that point you're not repeating a build. You're building a new image.

Re: Ask HN: Who operates at scale without containers?

#406

Post by DeathArrow »

Right now I don't see a good alternative to containers and orchestration. At my last workplace we developed a microservice based apps which run in Kubernetes. It was running smooth, it was easy to load balance, easy to scale and we had a very good availability.

At my current workplace we develop a microservice based app (kind of) but each service is running on the physical same server. No load balance, no failover and I don't want to know what the availability will be when it will be exposed to outside world.

Re: Ask HN: Who operates at scale without containers?

#407

Post by JeremyNT »

mountainriver wrote: post #246
zthrowaway wrote: post #138

Earlier quoted context omitted.

People still think stateful things are impossible on k8s but Stateful sets and persistent volumes solves a lot of this. You should be relying on out of the box DB replication to make sure data is available in multiple areas. This is no different on other platforms.

Yes you can run DBs on kube now, much of people thinking this isn’t good comes from years back when it wasn’t

Does it buy you much though? The big issue with databases is storage, and you need local storage for optimal performance.

You can ask k8s for a persistent volume on local storage, but at this point you have to treat that pod exactly the same way you'd treat a snowflake database server, because the local storage is what ultimately matters.

You can replicate to other pods with their own persistent volumes, but the replication overhead adds up too both in terms of performance and complexity.

I know there's stuff like vitess and crunchy data, which try to abstract this away, but the amount of layers buried here is very high for benefits that seem kind of nebulous compared to "big ol server" mode.

Re: Ask HN: Who operates at scale without containers?

#408

Post by tomc1985 »

iamstupidsimple wrote: post #397
tomc1985 wrote: post #259

Earlier quoted context omitted.

It's outright naked ageism. I have been studying a different trade, completely unrelated to software engineering or tech, and by far the weirdest thing is reading books or watching seminars from 20 or 30 years ago that are still relevant . How much of technology writing has that honor? Very little. Like I have been obsessed with tech and computers my whole life, I studied computer stuff when all the other kids were o…

The vast majority of computer science lectures from 20 years ago are still relevant. A ton has changed in terms of frameworks but the fundamentals are mostly the same.

One other point (since I can't edit): none of the used bookstores I am aware of take old technical books. Even the good ones (because they don't know and just see another tome that will likely sit on a shelf and collect dust)

Re: Ask HN: Who operates at scale without containers?

#409

Post by freedomben »

richardwhiuk wrote: post #232
freebuju wrote: post #174

Earlier quoted context omitted.

All the so-called "docker killers" are essentially unfinished products. They don't compare 1:1 to docker in feature set and even if they run as rootless, they still are vulnerable to namespace exploits in the Linux kernel. Though docker runs as root, it's still well protected out-of-the-box for the average user and is a very mature technology.

Kubernetes has removed docker, so I think that's basically it from a large scale perspective.

Yep. Mantis has announced intent to continue maintaining the docker shim (which allows k8s to talk to docker programmatically, but I can't imagine many people switching the default to docker unless they are manually installing k8s on their nodes, which used to be common but no longer is.

Re: Ask HN: Who operates at scale without containers?

#410

Post by evanelias »

nijave wrote: post #400
evanelias wrote: post #359

Earlier quoted context omitted.

> There's crazy things like this https://blogs.oracle.com/mysql/post/circular-replication-in-... that would be fairly difficult to run in a StatefulSet. This isn't a great example to cite, since traditional circular replication in MySQL is a massive anti-pattern... it's incredibly fragile and pretty much has no valid use-case. That's especially true today when other options like Galera or Group Replication are availa…

Probably a bad example, but the sentiment was: encapsulating years of DBA knowledge and complex enterprise architectures in generic k8s abstractions can be incredibly difficult. For instance, you put a RDBMS in a StatefulSet. You figure out how to replicate between pods. Now a piece of hardware fails and a replica needs rebuilt--what does that? So you add an operator into the mix. What happens if bin logs have been p…

Oh yes to be clear I definitely agree with the overall sentiment. I just wouldn't ever cite circular replication as an example. Basically saying "it's hard to do [inherently flawed terrible thing] on k8s!" just detracts from the argument :)

fwiw my team eventually automated pretty much "everything" for Facebook's database fleet, and it was hundreds of thousands of lines of custom automation code (non-k8s), many years of work for 10+ extremely experienced engineers. In a k8s environment I suspect it would have been even more work.

The open source mysql k8s operators are certainly getting better with time, but there's still a lot of stuff they don't handle.

Post reply on HN