I could have better explained what I was talking about. The point of a custom kernel would be to get extremely fast startup and context switch times, and extremely low per-process memory overhead, by doing things that Linux doesn't do - either because they're unnecessary or unhelpful for typical Linux use cases, or because they require making stronger assumptions about what userland needs to do. As a relatively flashy example, if you can assume that processes don't need overlapping addresses, you can share page tables between processes and rely on PCID/ASID to ensure each process can only access its own memory. [3] But there are also simpler things like - on Linux, starting a process requires forking the parent process, creating copies of its page tables, only for `exec` to immediately throw them away. Even posix_spawn is implemented on top of vfork. That's fast enough for Linux, but it's unnecessary. A custom kernel that doesn't provide Unix compatibility could dispense with it.
To be fair, none of that may be necessary. This blog post is advertising 1-2ms startup time. Based on a little benchmark [1] I just ran, on my desktop, Linux can execute a trivial process (/bin/busybox true) from start to finish in 0.4ms. A real application executed normally would obviously take much longer to start up. But I think their 1-2ms number is based on snapshotting the process after initialization, so a fair comparison would do that on native as well. Full process snapshotting for native processes is something that has been done before ('unexec', CRIU) but admittedly isn't very common, and I don't have a good way to test it. With snapshotting, there won't be any code executing in the process at startup, so most of the initialization time would be eliminated, but the mere act of loading more memory into the page table would have some cost. But probably not very much.
However, Fermyon uses Wasmtime, and Wasmtime's blog post on snapshotting [2] advertises single-digit microsecond startup, which is pretty cool. Starting native processes in single-digit microseconds is out of the question under Linux. But a custom kernel might be able to get there.
Edit: I guess I should add something. I'm not saying a custom kernel should just be table stakes. Writing one is obviously a major commitment and requires a high level of expertise. But you could say the same thing about JIT runtimes such as Wasmtime. There is even some overlap between what they're doing (i.e. loading binaries and managing processes). A kernel and a JIT are still fairly different projects, but they both require similar types of expertise, and I think the level of complexity is also comparable.
[1] https://gist.github.com/comex/0402dc95afb1c2ca3076d5b1b64bcc...
[2] https://bytecodealliance.org/articles/wasmtime-10-performanc...
[3] https://stackoverflow.com/questions/73753466/how-does-linux-...