Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Bugs Rust won't catch

corrode.dev

Post reply on HN
Print view View original

381–390 of 395 posts

Re: Bugs Rust won't catch

#381

Post by ragall »

dapperdrake wrote: post #364
ragall wrote: post #354

Earlier quoted context omitted.

No, any remote system would have the same problem if one expected to use it as if it were local.

Not quite. For persistence latency, yes. For read-only access there could be way better caching, especially for common use cases like listing the contents of a filesystem directory. But stuff like this was excluded on purpose. NFS is really stupid. NFS made the assumption that a distributed system with over 100 times the latency of a local system could be treated like a local system in every single way.

It's wasn't "NFS", it was always the users that made that mistake. NFS can be used in a proper and productive manner, but it requires adjustments.

Re: Bugs Rust won't catch

#382

Post by ragall »

thesuperbigfrog wrote: post #378
ragall wrote: post #355

Earlier quoted context omitted.

New things can be made optional and tested outside production, and should not be rolled out in an LTS edition.

>> should not be rolled out in an LTS edition. The Rust coreutils were first rolled out in the previous release which was not LTS: https://canonical.com/blog/canonical-releases-ubuntu-25-10-q...

That was for 6 months only and in an intermediate version that many don't update to, especially in production environments.

Re: Bugs Rust won't catch

#384

Post by satvikpendem »

ozgrakkurt wrote: post #376
satvikpendem wrote: post #195

Unrelated but also in the category of bugs Rust won't catch (natively), there are crates that allow C++ style contracts, or more generally, dependent typing and can be used to catch issues at compile time rather than runtime. I use this one, anodized. https://docs.rs/anodized/latest/anodized/

What do you think about the mental load and ergonomics this brings into the code? Also compilation time increase?

There is some compile time increase but it brings a lot more guarantees to the code. There was a recent post by a Rust maintainer that he wanted to bring Rust closer to a theorem prover so that as many things as possible can be caught during compile time over time time which might be more disastrous.

Re: Bugs Rust won't catch

#385

Post by Joker_vD »

Arch-TK wrote: post #380
Joker_vD wrote: post #337

Earlier quoted context omitted.

Obviously, if you tie the ability to handle those things to your filesystem layout, either by loading dynamic libraries from whatever is /usr/lib, or by reading /etc/whatever.conf, or even providing a whole virtual mount à la /proc, chroot'ing gives you both with the ability to override the system-wide policy for yourself (pretty reasonable for DNS lookups, kinda dubious for username lookups) and the opportunity to a…

What alternative solution do you actually propose here? To clarify, the kernel doesn't (well, it gets complicated with things like NFSv4 but let's just ignore that since it doesn't really make or break my point) know anything about users or groups. It _only_ has the ability to manage some integers on a per-process basis and tie those into integers attached to files. Assuming for now that we want to keep this like tha…

> What alternative solution do you actually propose here?

I don't, I am just saying that if you do things like e.g. user/group name resolution outside of the kernel, then they will be done in the userspace (duh) with all of the implications (crossing the trust boundaries, and philosophical discussion on what "the system [database/service/X]" even is, etc) that come with it. Doing it in the kernel has other implications, like how Windows does its SID lookups.

> what other options do you have?

Depends on whether or not you want to have the user namespaces, and how you want things across the namespace boundaries to interact. If you want to have /etc/passwd (or a more complex apparatus) of the core system to always be in effect, with no ability to override it with e.g. chroot(2) — then you can do it, with some help from the kernel side but that's both the silly idea in itself, and the kernel folks would never agree to add a do_rpc_with_global_secured_system_service(2) to the kernel anyway.

> Even if the kernel did have the capacity to maintain this information, something would need to load it or back it

During the system startup, a daemon is launched that registers itself as a "global, secured system service" providing the "resolve user/group names" functionality with the kernel. Now everyone can interact with it with the do_rpc_with_global_secured_system_service() syscall (the kernel passes the request to this specific process and routes the replies back), and there is no way to override this interaction; this service is a global singleton. I could spend pages on writing why this is a stupid idea, and I'm sure you know them as well — but it's doable, just not in the spirit of Linux at all.

P.S. A tangent, but I am always amused when people explain that execve(2) can't do PATH lookup for its first argument because the kernel doesn't have access to the process's environment variables. Of course it has, it's the third parameter! Linux just doesn't do any name resolution in the kernel, period, there is glibc for that.

Re: Bugs Rust won't catch

#386

Post by Arch-TK »

Joker_vD wrote: post #385
Arch-TK wrote: post #380

Earlier quoted context omitted.

What alternative solution do you actually propose here? To clarify, the kernel doesn't (well, it gets complicated with things like NFSv4 but let's just ignore that since it doesn't really make or break my point) know anything about users or groups. It _only_ has the ability to manage some integers on a per-process basis and tie those into integers attached to files. Assuming for now that we want to keep this like tha…

> What alternative solution do you actually propose here? I don't, I am just saying that if you do things like e.g. user/group name resolution outside of the kernel, then they will be done in the userspace (duh) with all of the implications (crossing the trust boundaries, and philosophical discussion on what "the system [database/service/X]" even is, etc) that come with it. Doing it in the kernel has other implicatio…

I mean, regardless of how you set this up, at some point you're going to want to namespace things. The core problem here is that chrooting creates an imperfect namespace boundary (it's not really even intended as that).

Maybe we're agreeing, I'm not quite sure. But rather than having a separate concept of mapping IDs to names, you really want a unified concept of both, so that when you namespace it, you can't namespace half of it by accident.

Moreover, you want to ideally make it impossible to only namespace it and not anything which is affected by it. e.g. the filesystem where permissions are governed by user and group IDs.

The way this is currently done is with UID/GID mapping, which unless it is a 1:1 mapping to the current user, has to be done with privileges. But this restriction is a hint that the abstraction is bad.

Really what you'd want is to have the ability to also include the concept of "file owner" in the umbrella of "user namespace", so that you can have a setup where files can have arbitrary attributes settable by a user to specify that they're owned by "root" inside the namespace but by outside it.

I'd say the problem is a fundamental design issue at multiple levels of the stack, exacerbated by the need to maintain backwards compatibility.

Plan9 solves this entire problem (albeit with its own trade-offs) by unifying all abstractions in the filesystem (but nothing, of course, stops you from unifying things via some IPC/RPC/whatever protocol, the filesystem in Plan 9 is just a mount of a 9P fileserver).

You literally can't get into a situation like the one described here.

Re: Bugs Rust won't catch

#387

Post by dapperdrake »

uecker wrote: post #369
dapperdrake wrote: post #364

Earlier quoted context omitted.

Not quite. For persistence latency, yes. For read-only access there could be way better caching, especially for common use cases like listing the contents of a filesystem directory. But stuff like this was excluded on purpose. NFS is really stupid. NFS made the assumption that a distributed system with over 100 times the latency of a local system could be treated like a local system in every single way.

I am not sure why this means why "NFS is really stupid" if the user assumes that a distributed file system can be treated just like a local system. That is provides the same interface is what makes NFS extremely useful.

And also, this is what makes NFS useless.

Latency is at least two orders of magnitude higher. That is the (relevant) difference here. And treating it like a loc system with all the incidental non-optimizations made the NAS use-case take 40 hours for colored "ls" output.

Re: Bugs Rust won't catch

#388

Post by dapperdrake »

ragall wrote: post #381
dapperdrake wrote: post #364

Earlier quoted context omitted.

Not quite. For persistence latency, yes. For read-only access there could be way better caching, especially for common use cases like listing the contents of a filesystem directory. But stuff like this was excluded on purpose. NFS is really stupid. NFS made the assumption that a distributed system with over 100 times the latency of a local system could be treated like a local system in every single way.

It's wasn't "NFS", it was always the users that made that mistake. NFS can be used in a proper and productive manner, but it requires adjustments.

Which all boil down to "replace NFS with something that has a better data model."

Re: Bugs Rust won't catch

#389

Post by uecker »

dapperdrake wrote: post #387
uecker wrote: post #369

Earlier quoted context omitted.

I am not sure why this means why "NFS is really stupid" if the user assumes that a distributed file system can be treated just like a local system. That is provides the same interface is what makes NFS extremely useful.

And also, this is what makes NFS useless. Latency is at least two orders of magnitude higher. That is the (relevant) difference here. And treating it like a loc system with all the incidental non-optimizations made the NAS use-case take 40 hours for colored "ls" output.

I find it extremely useful and it works well for many use cases. This already implies that "it is useless" is pure nonsense. If it does not work for your usecase, just don't use it.

Re: Bugs Rust won't catch

#390

Post by ragall »

dapperdrake wrote: post #388
ragall wrote: post #381

Earlier quoted context omitted.

It's wasn't "NFS", it was always the users that made that mistake. NFS can be used in a proper and productive manner, but it requires adjustments.

Which all boil down to "replace NFS with something that has a better data model."

Any remote data system would have the same problems. Looping over a list and synchronously fetching files one by one is equally foolish for S3 too.
Post reply on HN