Is using CUDA inside a container still a massive PITA?
Ask HN: Who operates at scale without containers?
61–70 of 446 posts
Re: Ask HN: Who operates at scale without containers?
#62My company runs without containers. We process petabytes of data monthly, thousands of CPU cores, hundreds of different types of data pipelines running continously, etc etc. Definitely a distributed system with lots of applications and databases. We use Nix for reproducible builds and deployments. Containers only give reproducible deployments, not builds, so they would be a step down. The reason that's important is t…
I asked the question without context because I didn't want to fire the thread off in the wrong direction but I suppose in a comment chain it's fine. For what it's worth we do use containers and K8s heavily at my current job.
I know that there are quite a few people opposed to the state of containers and the technologies revolving around them. I don't think the arguments they present are bad. [Attacking the premise](https://drewdevault.com/2017/09/08/Complicated.html) isn't particularly hard. What I don't see a lot of, however, is alternatives. "Learn how to do ops" is not exactly pertinent when most documentation on the subject will point you towards containers.
In addition the whole principle, while clearly proven, does strike me as a patch on existing platforms and executable formats that weren't designed to solve the sorts of problems that we have today. While efforts could have been made to make software universally more portable it seems we opted for finding the smallest feasible packaging method with already existing technology and have been rolling with it for a decade.
So essentially I'm interesting in knowing how people find ways to reproduce the value these technologies offer, what they instead rely on, which things they leave on the table (eg "we work with petabytes of data just fine but deploying updates is a nightmare"), how much manual effort they put into it, etc. New greenfield projects attempting to replace the fundamentals rather than re-orchestrate them are also very pertinent here.
Re: Ask HN: Who operates at scale without containers?
#63My company runs without containers. We process petabytes of data monthly, thousands of CPU cores, hundreds of different types of data pipelines running continously, etc etc. Definitely a distributed system with lots of applications and databases. We use Nix for reproducible builds and deployments. Containers only give reproducible deployments, not builds, so they would be a step down. The reason that's important is t…
I would love to hear more about your architecture and deployment... do your services run as NixOS modules? Are you using NixOps or Morph or something else? How does your world look like without K8s and Containers?
Re: Ask HN: Who operates at scale without containers?
#64My company runs without containers. We process petabytes of data monthly, thousands of CPU cores, hundreds of different types of data pipelines running continously, etc etc. Definitely a distributed system with lots of applications and databases. We use Nix for reproducible builds and deployments. Containers only give reproducible deployments, not builds, so they would be a step down. The reason that's important is t…
> Containers only give reproducible deployments, not builds Could someone elaborate on this please? Doesn't it depend entirely on your stack how reproducible your build is? Say I have a Python app with its OS level packages installed into a (base) image and its Python dependencies specified in a Pipfile, doesn't that make it pretty reproducible? Is the weak spot here any OS dependencies being installed as part of the…
> Containers only give reproducible deployments, not builds
I think that means containers alone are insufficient for creating reproducible builds, not that containers make reproducible builds impossible.
Re: Ask HN: Who operates at scale without containers?
#65I worked at WhatsApp, prior to moving to Facebook infra, we had some jails for specific things, but mostly ran without containers. Stack looked like: FreeBSD on bare metal servers (host service provided a base image, our shell script would fetch source, apply patches, install a small handful of dependencies, make world, manage system users, etc) OTP/BEAM (Erlang) installed via rsync etc from build machine Application…
That's a very cool deployment method to just use rsync like that! Simple and very composable. And now I feel self-conscious about my pile of AWS and Docker!
It's 'easy' to do it with rsync and Make as long as you don't have a lot of dependencies. Once you start pulling in dependencies, you have to figure out how to get them installed, hopefully before you roll code that uses them. If the dependencies are small, you can pull them into your source (more or less) and deploy them that way, but that may make tracking upstream harder. (otoh, a lot of things I personally used like that were more of pull this probably abandoned thing that's close to what we need, and modify it quite a bit to fit our needs; there wasn't a need or desire to follow upstream after that)
Re: Ask HN: Who operates at scale without containers?
#66Back in 2010 I built and operated MySpace' analytics system on 14 EC2 instances. Handled 30 billion writes per day. Later I was involved in ESPN's streaming service which handled several million concurrent connections with VMs but no containers. More recently I ran an Alexa top 2k website (45 million visitors per month) off of a single container-free EC2 insurance. Then I spent two years working for a streaming compa…
Docker is far heavier - the overhead is the flexibility and process isolation you get. I imagine that's really useful for certain types of workloads (e.g. an ETL pipeline), but is crazy inefficient for something single purpose like a web app.
Re: Ask HN: Who operates at scale without containers?
#67My company runs without containers. We process petabytes of data monthly, thousands of CPU cores, hundreds of different types of data pipelines running continously, etc etc. Definitely a distributed system with lots of applications and databases. We use Nix for reproducible builds and deployments. Containers only give reproducible deployments, not builds, so they would be a step down. The reason that's important is t…
> Containers only give reproducible deployments, not builds Could someone elaborate on this please? Doesn't it depend entirely on your stack how reproducible your build is? Say I have a Python app with its OS level packages installed into a (base) image and its Python dependencies specified in a Pipfile, doesn't that make it pretty reproducible? Is the weak spot here any OS dependencies being installed as part of the…
The dependencies can change out from underneath you transparently unless you pin everything all the way down the stack. Upstream docker images for example are an easy to understand vector of change. The deb packages can all change minor versions between runs, the npm packages (for example) can change their contents without making a version bump. Theres tons of implicit trust with all these tools, build wise.
Re: Ask HN: Who operates at scale without containers?
#68Is using CUDA inside a container still a massive PITA?
Can you elaborate on what makes containerized CUDA so difficult?
Re: Ask HN: Who operates at scale without containers?
#69My company runs without containers. We process petabytes of data monthly, thousands of CPU cores, hundreds of different types of data pipelines running continously, etc etc. Definitely a distributed system with lots of applications and databases. We use Nix for reproducible builds and deployments. Containers only give reproducible deployments, not builds, so they would be a step down. The reason that's important is t…
> But I'd so often dive in to understand the details and be disillusioned about how much complexity there is for relatively little benefit.
I feel similarly about many cloud services in e.g. AWS. I get where it's sometimes handy, but the management overhead can get pretty insane. I used to hear people say "just do X in AWS and ta-da you're done!" only to find out that in many cases it isn't actually a net improvement until you are doing a lot of the same thing. And then it costs $$$$$$.
Re: Ask HN: Who operates at scale without containers?
#70A previous company I was at we built a new AMI for each prod release, and used EC2 auto scaling groups. I much prefer k8s, but that method worked fine since we were already vendor-locked to AWS for other reasons.
I'm not sure what you mean by:
> without relying on the specific practice of shipping software with heaps of dependencies.
Do you mean like Heroku? That usually gets expensive really quickly as you scale.