Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Who operates at scale without containers?

news.ycombinator.com

Post reply on HN
Print view View original

271–280 of 446 posts

Re: Ask HN: Who operates at scale without containers?

#271

Post by dqv »

charcircuit wrote: post #249
Ericson2314 wrote: post #202

Earlier quoted context omitted.

Yeah Nix is why "choose boring" is a bit of a misnomer. The truth is tautologically unhelpful: choose good technologies. Most people have trouble separating snake oil and fads from good things, so old/boring is a safe heuristic. Nix is not boring, but it is honest. It's hard because it doesn't take half measures. It's not an easy learning curve gaslighting to you to a dead end. But taste in choosing good technologies…

Nix isn't a good option for developing rails applications. The nix ecosystem doesn't have good support for ruby meaning you will waste a lot of time and run into a lot of issues getting stuff to work. That's not boring. An example of being boring would be using Nix for something like C where the ecosystem around it is already built up. I was really pro Nix at the time I took on a rails project and it caused many prob…

What was the problem you were having with Ruby (on Rails) and Nix? It wasn't ergonomic for me, but I was able to get things to work.

Re: Ask HN: Who operates at scale without containers?

#272

Post by secondcoming »

Nextgrid wrote: post #2

I've been at a company where they weren't (yet) using containers nor K8S. The build process would just create VM images with the required binaries in there and then deploy that to an autoscaling group. It worked well and if you only ever intend to run a single service per machine then is the right solution.

My company works this way.

Our Jenkins creates a debian package, and deploying involves installing this package on a machine and creating an image. This image is then deployed to the ASGs. We operate in 5 regions in GCP.

Re: Ask HN: Who operates at scale without containers?

#273

Post by Thaxll »

smilliken wrote: post #189
Thaxll wrote: post #84

Earlier quoted context omitted.

The thing is, you spent probably a lot of time on things that were granted elsewhere and while the rest of the world is improving on those tech you keep your home grown solution that is harder and harder to maintain. Plus the knowledge that is not transferable. > Containers only give reproducible deployments, not builds, so they would be a step down. This is not true, if you use a docker image A with specific version…

Dan Luu wrote an essay on this topic recently: https://danluu.com/nothing-works/ He ponders why it is that big websites inevitably have kernel developers. Way out of their domain of expertise, right? If you adopt a technology, you're responsible for it. When Kubernetes inevitably has an issue that is a blocker for us, I don't have confidence in my ability to fix it. When an internal python or shell program has an iss…

> When Kubernetes inevitably has an issue that is a blocker for us, I don't have confidence in my ability to fix it. When an internal python or shell program has an issue that is a blocker for us, I change it.

I doubt that this happen in reality because Kubernetes cover uses cases for pretty much everyone, now I would doubt the knowledge of a regular dev to try to mimic a solution that k8s already does.

Re: Ask HN: Who operates at scale without containers?

#274

Post by Thaxll »

zomglings wrote: post #155
Thaxll wrote: post #84

Earlier quoted context omitted.

The thing is, you spent probably a lot of time on things that were granted elsewhere and while the rest of the world is improving on those tech you keep your home grown solution that is harder and harder to maintain. Plus the knowledge that is not transferable. > Containers only give reproducible deployments, not builds, so they would be a step down. This is not true, if you use a docker image A with specific version…

The OP is talking about reproducible builds . A docker image is a build artifact . If I gave you a Dockerfile for a node.js service that installed its dependencies using npm, and the service used non-trivial dependencies, and I gave you this file 10 years after it was first written, chances are you would not be able to build a new image using that Dockerfile. This would be a problem for you if you had to make some sm…

Docker is also used to build things not just run them, multistage build with dependency vendoring, you have a 100% reproducible build.

It's very easy to do in Go.

You pin the version used to build your app ( the runtime version ) In your git repo you have dependcies

That's it.

Re: Ask HN: Who operates at scale without containers?

#275

Post by remram »

jandrewrogers wrote: post #131
Thaxll wrote: post #84

Earlier quoted context omitted.

The thing is, you spent probably a lot of time on things that were granted elsewhere and while the rest of the world is improving on those tech you keep your home grown solution that is harder and harder to maintain. Plus the knowledge that is not transferable. > Containers only give reproducible deployments, not builds, so they would be a step down. This is not true, if you use a docker image A with specific version…

Sure, Kubernetes can run a database, but not efficiently. Companies with intensive data infrastructure frequently operate at scale without containers, either VMs or bare metal. The larger the data volume, the less likely they are to use containers because efficiency is more important. It is also simpler to manage this kind of thing outside containers, frankly, since you are running a single process per server. People…

Running on Kubernetes doesn't mean all data has to be on network disks.

Re: Ask HN: Who operates at scale without containers?

#276

Post by eatonphil »

onebot wrote: post #90

We use freebsd jails and a lightweight in house orchestration tool written in Rust. We are running hundreds of Ryzen machines with 64 cores. Our costs compared to running equivalent on Amazon is so much less. We estimate our costs are about 6x lower than AWS and we have far better performance in terms of networking, CPU, and disk write speed. Jails has been a pleasure to work with! We even dynamically scale up and do…

What company is this? And are you hiring? Asking for a friend.

Re: Ask HN: Who operates at scale without containers?

#277

Post by deathanatos »

smilliken wrote: post #35

My company runs without containers. We process petabytes of data monthly, thousands of CPU cores, hundreds of different types of data pipelines running continously, etc etc. Definitely a distributed system with lots of applications and databases. We use Nix for reproducible builds and deployments. Containers only give reproducible deployments, not builds, so they would be a step down. The reason that's important is t…

> Kubernetes has mostly focused on stateless applications so far. That's the easy part! The hard part is managing databases.

Kubernetes can absolutely host stateful applications. A StatefulSet is the app-level construct pretty much intended for exactly that use case. My company runs a distributed database on top of Kubernetes.

(We have another app that uses StatefulSets to maintain a cache across restarts, so that it can come up quickly, as it otherwise needs to sync a lot of data. But, it is technically stateless: we could just sync the entire dataset, which is only ~20 GiB, each time it starts, but that is wasteful, and it makes startup, and thus deployments of new code, quite slow. It would also push the limits of what an emptyDir can accommodate in our clusters' setup.)

(The biggest issue we've had, actually, with that, is that Azure supports NVMe, and Azure has AKS, but Azure — for God only knows what reason — apparently doesn't support combining the two. We'd love to do that, & have managed k8s & nice disks, but Azure forces us to choose. This is one of my chief gripes about Azure's services in general: they do not compose, and trying to compose them is just fraught with problems. But, that's an Azure issue, not a Kubernetes one.)

Re: Ask HN: Who operates at scale without containers?

#278

Post by wswope »

onebot wrote: post #90

We use freebsd jails and a lightweight in house orchestration tool written in Rust. We are running hundreds of Ryzen machines with 64 cores. Our costs compared to running equivalent on Amazon is so much less. We estimate our costs are about 6x lower than AWS and we have far better performance in terms of networking, CPU, and disk write speed. Jails has been a pleasure to work with! We even dynamically scale up and do…

As a tangent to this, I highly recommend https://bloom.host for anyone looking for just a few cores worth of Ryzen performance. They’re primarily a gaming server host, but their general-purpose VPS are absolutely stellar, and the support is top-notch. I tested them out after they were recommended on another HN thread, and got a 3x speedup based on a few heavy postgres queries I benchmarked on a $20/mo Digital Ocean instance vs. a $20/mo Bloom instance.

Data center Ryzen really knocks it out of the park.

Re: Ask HN: Who operates at scale without containers?

#279

Post by john-shaffer »

smilliken wrote: post #81
claytonjy wrote: post #47

Earlier quoted context omitted.

is Nix a hurdle when onboarding engineers? do only a few people need to know the language? I've wanted to use Nix and home-manager for personal stuff but the learning curve seems big.

In my case, adopting Nix was a response to having a poor onboarding process for new engineers. It was always fully automated, but it wasn't reliable before nix. So somebody would join the team, and it was embarrassing because the first day would be them troubleshooting a complex build process to get it to work on their machine. Not a great first impression. So I adopted Nix "in anger" and now new machines always buil…

Have you had problems with Nix on macOS? Nix works great for me, but I can't use it much to deal with installs or synchronize dependencies because the devs on macOS can't get Nix running.

Re: Ask HN: Who operates at scale without containers?

#280

Post by Ericson2314 »

charcircuit wrote: post #249
Ericson2314 wrote: post #202

Earlier quoted context omitted.

Yeah Nix is why "choose boring" is a bit of a misnomer. The truth is tautologically unhelpful: choose good technologies. Most people have trouble separating snake oil and fads from good things, so old/boring is a safe heuristic. Nix is not boring, but it is honest. It's hard because it doesn't take half measures. It's not an easy learning curve gaslighting to you to a dead end. But taste in choosing good technologies…

Nix isn't a good option for developing rails applications. The nix ecosystem doesn't have good support for ruby meaning you will waste a lot of time and run into a lot of issues getting stuff to work. That's not boring. An example of being boring would be using Nix for something like C where the ecosystem around it is already built up. I was really pro Nix at the time I took on a rails project and it caused many prob…

Please check out https://github.com/NixOS/rfcs/pull/109 !

I absolutely agree language-specific package managers need better support. Nix RFC 109 is a simple-stupid way to get some "important from derivation" in Nixpkgs, so we can collectively dogfood them "lang2nix" tools and finally make them good.

It is really sad to me that many users understandly think "hmm, C is crazy, surely this Nix should be even better with more uniform language ecosystems packages!", only to die on the hell of the lang2nixs all being just not quite good enough. This is huge stumbling block for developement shops that try to use Nix that I want to see removed!

Post reply on HN