Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Faster filesystem access with Directfs

gvisor.dev

Post reply on HN
Print view View original

41–50 of 58 posts

Re: Faster filesystem access with Directfs

#41

Post by predakanga »

Roark66 wrote: post #34
nine_k wrote: post #20

Earlier quoted context omitted.

A container, being basically a chroot, consumes a rather small amount of resources, mostly as space in namespace and ipfilter tables. If your containers use many of the same base layers (e.g. the same Node or Python image), the code pages will be shared, as they would be shared with plain OS processes. Running several processes in a container is the norm. First, you run with --init anyway, so there is a `tini` parent…

I quite like containers to limit/reserve the ram/cpu use for certain processes. For example imagine a tiny service used by Few concurrent users that needs a SQL db, a app server, and a reverse proxy (For ssl/caching) in front. I'm quite happy to put stuff like this on a tiny VM with 1vCPU and 1GB RAM. Mnthly cost ~$5 for compute. I typically reserve/limit 64MB/128MB for nginx, 384mb/512mb for mariadb, and 256mb/384mb…

Systemd has a mechanism[0] for configuring those limits.

I believe you can limit a unit to 1 vCPU and 256MB of memory by using something like the following:

[Service]

CPUQuota=100% # 100% of a core

MemoryLimit=256MB

Red Hat has some documentation[1] as well if the systemd stuff is too oblique.

[0]: https://www.freedesktop.org/software/systemd/man/systemd.res...

[1]: https://access.redhat.com/documentation/en-us/red_hat_enterp...

Re: Faster filesystem access with Directfs

#43

Post by intelVISA »

eyberg wrote: post #15
ec109685 wrote: post #12

I still don’t know why Google has gvisor and AWS has firecracker. Isn’t the firecracker approach strictly better than Google’s approach?

If you want to join us in the peanut gallery, AWS originally "adapted" Google's crosvm for firecracker. gVisor, if not using hw-backed virtualization, has absolutely horrendous performance because of, amongst other things, ptrace, which is one reason why this blogpost exists.

Outside of the peanut gallery we just roll our own VMM, VMX and friends is well established at this point why settle for a hacky impl.?

Re: Faster filesystem access with Directfs

#44

Post by watermelon0 »

Thaxll wrote: post #39
ec109685 wrote: post #12

I still don’t know why Google has gvisor and AWS has firecracker. Isn’t the firecracker approach strictly better than Google’s approach?

Firecracker does not work with long running process. It's only good for function as a service / serverless stuff.

What is your definition of long running processes?

AWS Fargate (containers as a service for ECS/EKS) uses Firecracker under the hood, and you can easily have the container up for weeks, and probably even for months.

Similarly, Fly.io also uses Firecracker, and again, you can have weeks/months long uptime on containers.

Re: Faster filesystem access with Directfs

#45

Post by amscanne »

fefe23 wrote: post #32

This is a step back. The reason to have this in a separate process is so it can be audited "to death" because the code base is small. gvisor itself is so big that doing an exhaustive audit is out of the question. Google has mostly switched to fuzzing because the code bases have all become too bloated to audit them properly. The reason you have gvisor is to contain something you consider dangerous. If that contained c…

The reason you can focus auditing on the second process is because you have a security architecture that enables that. Of course the security mechanisms you’re relying on there need to be exercised and occasionally fall apart too (meltdown, MDS, etc.).

Process isolation is not the only tool that you have to build a secure architecture. In this case, capabilities are still being limited by available FDs in the first process (as well as seccomp and the noting namespacing and file system controls), and access to FDs is still mediated by the second process. There is no such thing as “being able to take access away … to a file that was already opened” as this is simple not part of the threat model or security model being provided. You still need to be diligent about these security mechanisms as well.

The idea that Google has given up and just does fuzzing is nonsense. Fuzzing is a great tool, and has become more common and standardized — that’s all. It is being added to the full suite of tools.

Re: Faster filesystem access with Directfs

#46

Post by amscanne »

intelVISA wrote: post #43
eyberg wrote: post #15

Earlier quoted context omitted.

If you want to join us in the peanut gallery, AWS originally "adapted" Google's crosvm for firecracker. gVisor, if not using hw-backed virtualization, has absolutely horrendous performance because of, amongst other things, ptrace, which is one reason why this blogpost exists.

Outside of the peanut gallery we just roll our own VMM, VMX and friends is well established at this point why settle for a hacky impl.?

This sort of feels like seeing someone riding a bike and saying: why don’t they just get a car? The simple fact is that containers and VMs are quite different.

I’m responding to what I believe is the intent of the comment, but I will also point out that on a literal level it doesn’t make sense. Whether something uses VMX and friends or not is a red herring, as gVisor also “rolls it own” VMM [1] and certainly makes use of VMX and friends.

[1] https://github.com/google/gvisor/tree/master/pkg/sentry/plat...

Re: Faster filesystem access with Directfs

#47

Post by intelVISA »

amscanne wrote: post #46
intelVISA wrote: post #43

Earlier quoted context omitted.

Outside of the peanut gallery we just roll our own VMM, VMX and friends is well established at this point why settle for a hacky impl.?

This sort of feels like seeing someone riding a bike and saying: why don’t they just get a car? The simple fact is that containers and VMs are quite different. I’m responding to what I believe is the intent of the comment, but I will also point out that on a literal level it doesn’t make sense. Whether something uses VMX and friends or not is a red herring, as gVisor also “rolls it own” VMM [1] and certainly makes us…

Apologies my reply was toward Firecracker, I appreciate gVisor is a sandbox solution/KVM shim rather than a true VMM.

Re: Faster filesystem access with Directfs

#48

Post by amscanne »

intelVISA wrote: post #47
amscanne wrote: post #46

Earlier quoted context omitted.

This sort of feels like seeing someone riding a bike and saying: why don’t they just get a car? The simple fact is that containers and VMs are quite different. I’m responding to what I believe is the intent of the comment, but I will also point out that on a literal level it doesn’t make sense. Whether something uses VMX and friends or not is a red herring, as gVisor also “rolls it own” VMM [1] and certainly makes us…

Apologies my reply was toward Firecracker, I appreciate gVisor is a sandbox solution/KVM shim rather than a true VMM.

Ah, now it makes more sense to me. Thank you for the clarification.

Re: Faster filesystem access with Directfs

#49

Post by nomel »

Patrickmi wrote: post #29
nomel wrote: post #10

Earlier quoted context omitted.

> writes operation more security at risk than reads I think, in the context of security, this is like asking if it's worse to die by a car or die by a bus.

Lol at least one is recoverable

Security exists because of the meaning of the bits. If those bits represent credentials to your bank account, then "recoverable" hits different.

Re: Faster filesystem access with Directfs

#50

Post by dilyevsky »

topspin wrote: post #30
dilyevsky wrote: post #23

Earlier quoted context omitted.

In k8s that already exists via CSI[0] but kubelet is handling the setup/teardown signaling and it requires 3rd party provisioner daemon so higher level than container runtime (runsc in this case). [0] - https://kubernetes-csi.github.io/docs/

Yes. I know. K8s has delivered the moral equivalent of what we've had built-in to our OS kernels[1] since before some of the people reading this were born, and they've only had to add two layers of complexity, fragility and inscrutability on top of k8s itself, one of which is a third party dependency. This is my excited face. :| [1] 2005: https://lwn.net/Articles/131747/

No k8s has not delivered that. It's built an orchestration layer on top of iSCSI, NVMeOF or whatever "remote disk" tech the kernel has implemented and abstracted that from devs which was the whole point of k8s.
Post reply on HN