Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Rewriting Every Syscall in a Linux Binary at Load Time

amitlimaye1.substack.com

Post reply on HN
Print view View original

31–40 of 50 posts

Re: Rewriting Every Syscall in a Linux Binary at Load Time

#31

Post by qbane »

xelaboi wrote: post #14

You either have a writing style that is uncannily similar to what an LLM generates, or this article was substantially written by an LLM. I don't know what it is about the style, but I just find it a bit exhausting, like an overfit on "engaging writing" that strips away sincerity.

There is even a table copy-pasted into a paragraph without noticing.

> What’s needed is something different:

> Requirement ptrace seccomp eBPF Binary rewrite Low overhead per syscall No (~10-20µs) Yes Yes Yes [...]

Re: Rewriting Every Syscall in a Linux Binary at Load Time

#35

Post by amitlimaye »

jacobgorm wrote: post #29
rep_lodsb wrote: post #17

Earlier quoted context omitted.

Yes, that seems unneccessary. The overhead of trapping and rewriting every syscall instruction once can't be (much) greater than that required for rewriting them at the start either. Even if you disallow executing anything outside of the .text section, you still need the syscall trap to protect against adversarial code which hides the instruction inside an immediate value: foo: mov eax, 0xc3050f ;return a perfectly h…

I think the point here is optimizing for the common case, the untrusted code is still running inside a VM, so you can still trap malicious or corner cases using a more heavy-handed method. The blog post does mention "self-healing" of JIT-generated code for instance. It is possible to restrict the call-flow graph to avoid the case you described, the canonical reference here is the CFI and XFI papers by Ulfar Erlingsso…

The follow on posts describe where I plan to run the binaries. the idea is to run in a guest with no kernel and everything running at ring 0 that makes the sysret a dangerou thing to call. we don't have anything running at ring 3 also the syscall instruction clobber some registers all in all between the int3 and syscall instruction i counted around 20 extra instructions in my runtime. ( This is a guess me trying to figure what would happen). That is why the int3 becomes faster for what i am trying to build. The toolchain approach suffers from the diversity of options you have to support even if ignore stuff you guys encountered. Might be easier with llvm based things but still too many things to patch and the movement you tell people used my build environment it meets resistance. I am currently aiming for python which is easy to do. The JIT is when i want to do javascript which i keep pushing out because once i go down there i have to worry about threading as well. Something i want to chase but right now trying to get something working.

Re: Rewriting Every Syscall in a Linux Binary at Load Time

#36

Post by amitlimaye »

twic wrote: post #27
ghoul2 wrote: post #23

Earlier quoted context omitted.

Isn't that exactly what gvisor does?

Yes: https://gvisor.dev/docs/

gvisor tries to be a complete kernel in userland we are not trying to. We will consciously choose never to try and support multi-proess env in the sandbox. The idea is there are enough people running single process containers and they can benefit from a lighter more secure runtime. This solution will not try to replace the kernel. For example the python tests we run for https to some website ends up runnign implementing only 60 syscalls not 350. i expect to add another 10-20 for support typescript but this will always be strictly single process.Plus the performance overhead of gvisor is substantial 2-10us ( me reading internet) for the system i am implemeting on the hot path it is less than 1us. Plus there is always the density story my shim currently is 4KB the python runtime is shared through memfd. I am working on a demo showing i can run 1000 vm on 512 MB ram each launching in under 30msec. Remember this will never replace or be able to handle generic mutli-process sandboxes this is targeted only at single process env where we can make lots of simplifying assumptions

Re: Rewriting Every Syscall in a Linux Binary at Load Time

#37

Post by monocasa »

coppsilgold wrote: post #7
monocasa wrote: post #6

Earlier quoted context omitted.

There's also SECCOMP_RET_USER_NOTIF, which is typically used by container runtimes for their sandboxing.

SECCOMP_RET_USER_NOTIF seems to involve sending a struct over an fd on each syscall. Do they really use it? Performance ought to suffer. Also gVisor (aka runsc) is a container runtime as well. And it doesn't gatekeep syscalls but chooses to re-implement them in userland.

They use a seccomp filter to decide which syscalls get sent to the other process for processing.

Re: Rewriting Every Syscall in a Linux Binary at Load Time

#38

Post by amitlimaye »

direwolf20 wrote: post #19
ozgrakkurt wrote: post #16

Earlier quoted context omitted.

I mean just distributing the regular compiled x86_64 binary and then running it as a normal executable on the client side but just using that syscall shim so it is safe.

If you think about the fundamentals involved here, what you actually need is for the OS to refuse to implement any syscalls, and not share an address space. A process is already a hermetically sealed sandbox. Running untrusted code in a process is safe. But then the kernel comes along and pokes holes in your sandbox without your permission. On Linux you should be able to turn off the holes by using seccomp.

seccomp is a very coarse filter and a very limited action set. think what you could do if you could see the payload of the syscall or change the output of a read syscall depending on agent identity.

Re: Rewriting Every Syscall in a Linux Binary at Load Time

#39

Post by amitlimaye »

JSR_FDED wrote: post #12

Love the detailed write up, thanks! This is the kind of foundation that I would feel comfortable running agents on. It’s not the whole solution of course (yes agent, you’re allowed to delete this email but not that email can’t be solved at this level)… let me know when you tackle that next :-)

AMA i am the author of that blog i have some working code just not something i want to share right away. Right now i am chasing density but yes security is something i will get to eventually. the issue is what to implement first :). This is the first of a series of blogs i am writing. you can check my substack. the next step is to show a density,launch speed demo hopefully middle of next week

Re: Rewriting Every Syscall in a Linux Binary at Load Time

#40

Post by amitlimaye »

jcalvinowens wrote: post #34
im3w1l wrote: post #11

What about int 80h?

Yeah, I had the same question. But I'd guess they probably disable IA32 completely.

Int80 is a great idea but int3 is what i landed on when i was looking and at this point just trying to get something working. The good thing about int80 is a 2 byte instruction i believe rather than int3 + nop that i am doing right now
Post reply on HN