Discover more from Matt Rickard
The ptrace syscall
ptrace (“process trace”) is a system call in Unix and Unix-like operating systems that intercepts system calls. It’s a powerful tool that enables tools like debuggers (e.g.,
gdb), reverse engineering tools, tracing, code injection, and even simple sandboxing. (see proot for an example of a
ptrace sandbox). The most interesting part of
ptrace is that you can do all of these things completely in user space (even sandboxing!).
Sandboxing. Roughly how a
ptrace sandbox works:
Fork a child process to run the untrusted code
ptracesyscall to trace the child process and intercept and monitor the child’s syscalls
Inspect the child process syscalls and arguments. Maybe enact some sort of security process, logging, modification, or something else.
Google’s gVisor can use ptrace as a backend for its sandboxing. It’s a lot more complicated than a vanilla
ptrace sandbox and works much more like User-mode Linux (UML).
strace (Linux) and
ktrace (macOS), which are userspace monitoring utilities that expose
ptrace functions as a binary. You can monitor or modify system calls for a running program. Solaris had
DTrace (by Bryan Cantrill).
Some other cool ideas you could do with
Time-travel debugging (record the whole state with
Transparent network traffic encryption (intercept
Resource limits and throttling (intercept
Live process migration or checkpointing (capture the entire state and copy somewhere else)
Custom process scheduling (intercept
Testing (fault injection, fuzzing, etc.)