51–58 of 58 posts
Earlier quoted context omitted.
"Direct" here is more analogous to the Direct as in DirectX and Direct3D.
directfs has nothing to do with DirectX. I find it deeply ironic this needs to be said here.
I still don’t know why Google has gvisor and AWS has firecracker. Isn’t the firecracker approach strictly better than Google’s approach?
Firecracker is hardware-based virtualization. gVisor is not virtualization at all but more like advanced sandboxing - it intercepts syscalls and proxies them on processeses behalf. That means gVisor is slower on i/o (which this new feature is trying to solve) but it also means it’s easier to implement and operate and you can run it in more environments (for examples in VMs where nested virtualization is not supported…
Earlier quoted context omitted.
Firecracker is hardware-based virtualization. gVisor is not virtualization at all but more like advanced sandboxing - it intercepts syscalls and proxies them on processeses behalf. That means gVisor is slower on i/o (which this new feature is trying to solve) but it also means it’s easier to implement and operate and you can run it in more environments (for examples in VMs where nested virtualization is not supported…
What are the reasons these days to not enable nested virtualization? I know AWS doesn’t.
Performance used to be a problem with nested virt but afaik both hw and software have caught up
Earlier quoted context omitted.
Yes. I know. K8s has delivered the moral equivalent of what we've had built-in to our OS kernels[1] since before some of the people reading this were born, and they've only had to add two layers of complexity, fragility and inscrutability on top of k8s itself, one of which is a third party dependency. This is my excited face. :| [1] 2005: https://lwn.net/Articles/131747/
No k8s has not delivered that. It's built an orchestration layer on top of iSCSI, NVMeOF or whatever "remote disk" tech the kernel has implemented and abstracted that from devs which was the whole point of k8s.
That may be the point, but the actual impact is "devs" became "devops" and now spend some multiple of their time actually developing software puzzling over operations abstractions.
Accessing local file systems from a container? What heresy is this? Containers must all be stateless webscale single-"process" microservices with no need of local file systems and other obsolescent concepts. Next thing you know someone will run as many as two whole "processes" in a container! Having dispensed with that bit of bitter sarcasm; solving their local filesystem performance/security problems is great and al…
There are other advantages — low fixed resource costs, global memory management and scheduling, no resource stranding, etc. — but the core intent of gVisor is to capture as many valuable semantics as possible (including the file system semantics) while adding a sufficiently hard security boundary.
I’m not saying moving the file system up into the sandbox is bad (which is basically what a block device gives you), just that there are complex trade-offs. The gVisor root file system overlay is essentially that (the block device is a single sparse memfd, with metadata kept in memory) but applied only to the parts of the file system that are modified.
This is a step back. The reason to have this in a separate process is so it can be audited "to death" because the code base is small. gvisor itself is so big that doing an exhaustive audit is out of the question. Google has mostly switched to fuzzing because the code bases have all become too bloated to audit them properly. The reason you have gvisor is to contain something you consider dangerous. If that contained c…
The reason you can focus auditing on the second process is because you have a security architecture that enables that. Of course the security mechanisms you’re relying on there need to be exercised and occasionally fall apart too (meltdown, MDS, etc.). Process isolation is not the only tool that you have to build a secure architecture. In this case, capabilities are still being limited by available FDs in the first p…
The old model howevwr was that read and write were translated to rpc calls to the broker. In that model you can take write access away even after you have given it to a process, because you have not actually given it. All writes still go through the broker process.
Earlier quoted context omitted.
The reason you can focus auditing on the second process is because you have a security architecture that enables that. Of course the security mechanisms you’re relying on there need to be exercised and occasionally fall apart too (meltdown, MDS, etc.). Process isolation is not the only tool that you have to build a secure architecture. In this case, capabilities are still being limited by available FDs in the first p…
As I understand it, the new model is that the process gets an opened fd passed by the broker and can then read and write to it as fd permissions allow. The old model howevwr was that read and write were translated to rpc calls to the broker. In that model you can take write access away even after you have given it to a process, because you have not actually given it. All writes still go through the broker process.
In the old model, reads/writes were not translated to RPCs. Only for regular files, the broker was donating FDs to the sentry (userspace kernel) and the sentry was allowed to perform read(2)/write(2) directly. This was done as a performance optimization long back.
What is different with directfs is that now the broker additionally donates FDs for other types of files as well (directories, sockets, etc) and the sandbox is allowed to operate on those FDs with more syscalls like mkdirat, symlinkat, etc. This drastically increases the independence of the sandbox is performing filesystem operations, so it does not need to invoke the broker via RPCs.
As described, the sentry is still constrained to operating on only the container filesystem via namespaces and other Linux security primitives.
Earlier quoted context omitted.
DirectStorage does, thought.
Ah... Okay, I think I see how the comment should have been read now...? I will blame whoever named directfs for using a confounding name one way or the other. :V
Point taken. In retrospect, directfs is not a good name as it gives the impression of a new filesystem implemention. I should have named this more along the lines of "direct access mode" as some of you have pointed out.
Thanks for the feedback.