Ask HN: Who operates at scale without containers? • Hacker News Board

Re: Ask HN: Who operates at scale without containers?

#431

Post by p_l » Thu, Mar 24, 2022, 8:22 AM UTC

qwertywert_ wrote: post #428

p_l wrote: post #393

Earlier quoted context omitted.

CRI-O is the target and containerd is the most common runtime implementing it at the moment.

Are you sure? This isn't my subject area but CRI-O looks like an alternative to containerd and implements the OCI compliant runtime like containerd does. And then there is a 3rd which is docker engine which is the one being dropped.

Sorry, I mixed up CRI and CRI-O. The roadmap is to remove dockershim (the interface to docker-compatible container runtime) and use only CRI - of which containerd and CRI-O are two compatible implementations.

Re: Ask HN: Who operates at scale without containers?

#432

Post by efreak » Thu, Mar 24, 2022, 9:37 AM UTC

wanderr wrote: post #426

mhitza wrote: post #37

Earlier quoted context omitted.

What a great service. I'd be curious if you could go into details how the radio feature worked back then, because I found myself receiving worse suggestions when I used similar features in Spotify/Google Play Music.

Oh man, I should write a blog post about that, as I built that feature myself. It was meant to be a stopgap until we could get some real matchine learning in there, but nothing else we tried did as well. First, for efficiency all recommendations were artist to artist,nl not song to song. That works well for a lot of genres but is pretty bad for others. We started with a free DB of artist similarities, I don't remembe…

I'm not aware of an MB similar artist database. I'm guessing you used music map[0], it's the only free database I know of for similar artists that doesn't require scraping.

[0]: https://www.music-map.com/

Re: Ask HN: Who operates at scale without containers?

#433

Post by alex_duf » Thu, Mar 24, 2022, 10:02 AM UTC

weego wrote: post #214

alex_duf wrote: post #45

Hey former Guardian employee here. The Guardian has hundreds of servers running, pretty much all EC2 instances. EC2 images are baked and derived from official images, similarly to the way you bake a docker image. We built tools before docker became the de facto standard, so we could easily keep the EC2 images up to date. We integrated pretty well with AWS so that the basic constructs of autoscaling and load balancer…

Also ex employee. Riff Raff is absolutely still an excellent mod for build and deploy. At the time I was there it the initial stack build via handwritten cloudformation script that was the friction and pain point.

Honestly I miss riff-raff :D

For anyone who doesn't know about it: https://github.com/guardian/riff-raff

Re: Ask HN: Who operates at scale without containers?

#434

Post by thealbert08 » Thu, Mar 24, 2022, 12:24 PM UTC

spicybright wrote: post #165

mateuszf wrote: post #157

Earlier quoted context omitted.

> Chose boring technology Doesn't fit in this case, I'd say Nix is still cutting edge. It may also be simple, but it's not easy (in the Rich Hickey sense).

Can you elaborate more? From what I've generally heard, Nix has been a good foundation for a lot of companies.

No post body was provided.

Re: Ask HN: Who operates at scale without containers?

#435

Post by dolmen » Thu, Mar 24, 2022, 12:38 PM UTC

motoboi wrote: post #80

freeqaz wrote: post #57

Earlier quoted context omitted.

That's a very cool deployment method to just use rsync like that! Simple and very composable. And now I feel self-conscious about my pile of AWS and Docker!

Containers are just tar.gz files, you know? The whole layers thing it’s just an optimization. You can actually very simply run those tar.gz files without docker involved, just cgroups. But then you’ll have to write some daemon scripts to start, stop, restart, etc Follow this path and soon you’ll have a (worst) custom docker. Try to create a network out of those containers and soon a (worst) SDN network appears. Try t…

Images are just tar.gz.

A container is a set of processes running from that image.

Re: Ask HN: Who operates at scale without containers?

#436

Post by dolmen » Thu, Mar 24, 2022, 12:46 PM UTC

Art9681 wrote: post #255

z3t4 wrote: post #228

Earlier quoted context omitted.

You can do a lot with just SystemD today. Would not get surprised if SystemD would eat up Docker too.

I'm personally interested in Unikernels. That tech seems to be a good contender for replacing K8s. Every app an appliance running in a hypervisor.

I even imagine a GOOS runtime for compiling a Go program that would talk directly to the hypervisor...

Many services I write use only network resources, no disk.

Re: Ask HN: Who operates at scale without containers?

#437

Post by dolmen » Thu, Mar 24, 2022, 12:50 PM UTC

toast0 wrote: post #65

freeqaz wrote: post #57

Earlier quoted context omitted.

That's a very cool deployment method to just use rsync like that! Simple and very composable. And now I feel self-conscious about my pile of AWS and Docker!

It's 'easy' to do it with rsync and Make as long as you don't have a lot of dependencies. Once you start pulling in dependencies, you have to figure out how to get them installed, hopefully before you roll code that uses them. If the dependencies are small, you can pull them into your source (more or less) and deploy them that way, but that may make tracking upstream harder. (otoh, a lot of things I personally used l…

rsync+make is the deployment+run.

But this assumes the source of rsync is already static.

Handling dependencies is done in a build process, upstream.

Fetching dependencies at deployment time is a direct path to deployment failures (especially if the dependency management relies on external resources or non-locked versionning).

Re: Ask HN: Who operates at scale without containers?

#438

Post by wanderr » Thu, Mar 24, 2022, 3:33 PM UTC

efreak wrote: post #432

wanderr wrote: post #426

Earlier quoted context omitted.

Oh man, I should write a blog post about that, as I built that feature myself. It was meant to be a stopgap until we could get some real matchine learning in there, but nothing else we tried did as well. First, for efficiency all recommendations were artist to artist,nl not song to song. That works well for a lot of genres but is pretty bad for others. We started with a free DB of artist similarities, I don't remembe…

I'm not aware of an MB similar artist database. I'm guessing you used music map[0], it's the only free database I know of for similar artists that doesn't require scraping. [0]: https://www.music-map.com/

Ah, I think you're right that it wasn't MB because I remember having to match artists by name rather than mbid. Music map doesn't sound familiar but my boss negotiated the access, I think I was just ingesting the data from a csv dump so it could have been anywhere.

Re: Ask HN: Who operates at scale without containers?

#439

Post by barbazoo » Thu, Mar 24, 2022, 5:49 PM UTC

kronin wrote: post #349

spion wrote: post #316

Earlier quoted context omitted.

Until there is a critical vulnerability in one of the components present on that image (system packages or application packages).

This is why I haven't adopted the practice of "Build your artifact along with the docker image that packages it". Instead, build your artifact and publish it to an artifact repository, just like we used to. _then_ wrap that artifact in a Docker image. Vulnerability found in the docker image? No problem. Build a new image with the same artifact.

I'm curious why you're being downvoted for this. Whoever disagrees, please share some context.

What you said about "wrapping" I interpret as: based on an image for instance with OS level dependencies you create another image with application level artifacts, e.g. a python application. When your app changes, you don't build the base image again, you only build the app image. This makes sense to me.

Re: Ask HN: Who operates at scale without containers?

#440

Post by hinkley » Thu, Mar 24, 2022, 7:08 PM UTC

dilyevsky wrote: post #230

meatmanek wrote: post #177

Earlier quoted context omitted.

pedantic nit-pick that doesn't detract from your main point: Turing completeness doesn't imply that it can never be idempotent. In fact we'd expect that a particular Turing machine given a particular tape should reliably produce the same output, and similarly any Turing-complete program given the exact same input should produce the exact same output. Turing machines are single-threaded, non-networked machines with no…

Yup, the problem is not that RUN is Turing Complete the problem is that it’s non-hermetic.

If it's Turing Complete you can't prove that it's hermetic. You can't even prove if it'll ever finish running (halting problem).

Maybe someday someone will invent a sort of EBPF for containers, but usually the first batch of RUN commands and the last are calling package managers, and in between you're doing things like creating users and setting permissions using common unix shell commands, which have the same problems.

With the possible exception of the unix package managers, we have no hope of those ever being rewritten inside of a proof system, because the package manager is often a form of dogfooding for the language community. The Node package manager is going to be written in Node and use common networking and archive libraries. Same for Ruby, Rust, you name it.