5.2 Kernel Runtime Patching
5.2.2 Configuration-specific Patching
In addition to patching the modules for external symbols as described above, the Linux (module) loader may also replace code with ar-chitecture and configuration-specific opcodes or code blocks that are only present in certain configurations and are replaced with no operation (NOP) instructions otherwise. In fact, such patching takes place in the kernel code as well. This is done to improve performance,
R untime Kernel Co de Integrit y
5.2. Kernel Runtime Patching
leverage special features such assymmetric multiprocessing (SMP) or (para-)virtualization on architectures that support them, or to accommodate debugging or tracing to provide extensibility. With these mechanisms the kernel can even replace entire functions. Such patching not only takes place in kernel modules, but in the kernel code as well.
In the following, we present the four cases in which such patching takes place in more detail. All of them have in common, that they patch an instruction and overwrite the rest of the available space withNOP instructions. The kernel unfortunately does not only use the straight forward 0x90 instruction as a NOP instruction. For performance reasons it also uses multi-byte instructions like: xchg ax, ax (0x60, 0x90) or nop dword [rax+rsi*4-0x40] (0xf, 0x1f, 0x44, 0x0, 0x0). Unfortunately, the exact multi-byte instructions used are also specific to the processor architecture and type currently used. Also, to patch code during runtime, the kernel does not need a special synchronization method. Instead, it first replaces the first byte of buffer with anint3 instruction, thus every CPU trying to execute this instruction will be trapped. Then the rest of the space is filled with the new content. As a last step the kernel replaces the first byte with the right instruction and notifies all waiting CPUs. To even further improve the performance, the kernel sets the interrupt handler function to the address directly after the patched buffer. Thus the int3 instruction is effectively changed into a NOP instruction.
5.2.2.1. Alternative Instructions
For certain functionalities, the specific opcodes used by the kernel vary depending on the available CPU’s feature set. Mostly, this approach is used when later CPU models support more efficient instructions. For every instruction sequence that should be replaced, a list of alternative instructions is provided with each kernel binary, ordered by the most preferred as last in the list. The Linux module loader replaces each instruction if the requested feature is supported by the CPU. This feature is mostly used, when a new CPU feature is
5. Runtime Kernel Code Integrity
introduced and the kernel implements support for the new feature. In this case, the kernel addsnopinstructions into the compiled binary code, that are then replaced by the required instructions during load-time (e. g. to enable/disable SMAP or to synchronize memory accesses). In other cases, the operand of a callorjmpinstruction is changed in order to execute different code during runtime. However, also some cases exist where entire kernel functions are replaced entirely with this feature. Notice that this is the only configuration-specific patching mechanism that is considered by Patagonix [57]. To be able to validate alternative instructions, we have to extract that specific information from the monitored guests memory or generate it based on the knowledge we have about the used hardware.
5.2.2.2. Hypercalls
Another situation in which code is commonly patched within the kernel is related to virtualization, in the form of hypercalls. Hyper-calls are analogous to system Hyper-calls, where a call is passed from a userland process to the kernel, in a virtualized environment which enable the guest system to interact with the hypervisor. Examples of such hypercalls include enabling and disabling interrupts, writing to specific processor registers, and MMU related functions.
As with relocation, hypercalls can not be inserted during compile time, as the kernel or its modules do not know if they are executed on real or virtualized hardware. Furthermore it is unknown which virtualization technology is used (e. g. XEN, KVM, etc.). Thus, the compiler provides hints in the compiled binary that a specific function (e. g. writing to the CR3 register, which holds the page tables for the currently executing process) is requested at a specific location in the kernel code. During load time the kernel is then able to decide how the requested function is provided and inserts the appropriate functionality. This can be done by inserting the corresponding opcodes directly, calling a function provided by the hypervisor, or even by jumping to a predefined location.
R untime Kernel Co de Integrit y
5.2. Kernel Runtime Patching f0 ff 00 lock inc DWORD PTR [rax]
3e ff 00 inc DWORD PTR [rax]
inc [rax]
lock prefix segment overwrite prefix
Figure 5.1.: Implementation of SMP in practice.
To facilitate this mechanism, the kernel maintains a table of alter-natives introduced by a hypervisor-specific driver that can be used as a replacement for a given instruction. This table must adhere to the kernel’sparavirt_patch_template interface. This feature is even employed in a non-virtualized or full-virtualized environment.
For full-virtualized environments, the kernel provides the native implementations for the requested functionality.
The kernel also suffers from another virtualization related problem.
To enable userspace applications to use the correct system call mech-anism the kernel maps a common page into every userspace process called the vdso page, containing functions that wrap the kernel’s current system call interface. Equal to this mechanism, a hypervisor can also map a custom page into the memory of a guest VM.
The patching mechanisms described so far are all conducted at load time. A common solution to these load time patching mechanisms is to take the initial hash after all modules have been loaded such that relocation is no longer an issue. However, this approach has a fundamental flaw: we cannot be sure that kernel code has not already been altered when the hash is calculated. Next we will describe patching mechanisms that are applied duringruntime.
5.2.2.3. Symmetric Multiprocessing Locks
SMP describes an architecture with multiple CPUs that share mem-ory. This is very common in modern PCs. There are portions of the kernel code that become critical sections (i. e. they share data that should only be accessed in an asynchronous, mutually exclusive
5. Runtime Kernel Code Integrity
fashion) and only when SMP and multiple CPU cores are enabled.
In this case, the critical section must be protected with locks. How-ever, in the interest of performance the kernel chooses to patch in these locks only if it recognizes it is operating in an environment in which more than one CPU is present. An example for such a modification is shown in Figure 5.1. This makes sense as the locking and unlocking operations are computationally expensive tasks and become unnecessary if only one CPU is active.
Furthermore, the Linux kernel supports enabling and disabling CPUs at runtime. This results in such patching also taking place at runtime. Note that adding and removing CPUs in a virtualized envi-ronment is frequently used for scalability. In addition to SMP locks it is conceptually also possible to patch arbitrary other instructions, in a fashion similar to alternative instructions, during runtime, once the number of active CPUs changes.
5.2.2.4. Jump Labels
The Jump Labels mechanism is used within the kernel to optimize
“highly unlikely” code branches to the point that their normal over-head is close to zero. Instead of checking whether a branch should be taken or not every time the control flow reaches a specific point, the kernel either replaces the conditional jump with a NOP instruction, and thereby omits the unlikely code, or with an unconditional jump to the unlikely code. Thus, the enabling/disabling of the function is an expensive task, but the runtime cost is completely avoided.
An illustration for a jump label is shown in Figure 5.2 on page 76.
Figure 5.2a shows the source code for a jump label in the kernel.
During normal execution, the jump label is deactivated. Thus the code in memory looks like represented by Figure 5.2b. Once the unlikely condition is fulfilled, the code in memory is replaced to the state shown in Figure 5.2c. Although this feature was mainly intended for debugging and tracing features, this mechanism is now frequently used both in the Linux scheduler and in the networking subsystem. For example, the latter uses it to activate and deactivate
R untime Kernel Co de Integrit y
5.2. Kernel Runtime Patching
certain netfilter hooks. The current state of each jump instruction is also maintained within a kernel data structure.
5.2.2.5. Function Tracing
Another mechanism that requires runtime code patching is the Ftrace function tracer. This tracer is mainly used to debug the kernel or measure performance. It is commonly called at the beginning of each function within the kernel or its modules. For performance reasons, each tracer call is replaced by a NOP slide when the feature is currently disabled. Although the feature is similar to the Jump Labels mechanism, its implementation is different and it depends on other kernel data structures. Consequently, both mechanisms must be considered separately.
5.2.2.6. Function Patching
In the meantime the kernel even supports live patching of entire kernel functions. This is to support security updates without the need to reboot the entire machine. In that case, the kernel either replaces an entire function with its new content, if the new version of that function is shorter then the original version, or the kernel uses the space that was reserved for the ftrace function tracer to insert an unconditional jump to the address of the new function.
5.2.3. Summary
In this section we provided the reader with an overview of the different patching mechanism that the Linux kernel uses. The shown mechanisms can be divided in two groups. The first group contains patches that only are applied during the loading of the kernel or its modules. The second category includes the mechanisms that are also used during runtime and thus are not precomputable. Table 5.1 contains a list of patching mechanisms for both categories.
5. Runtime Kernel Code Integrity
1 int function() {
2 [...]
3 if (unlikely(...)) {
4 doSomething();
5 }
6 [...]
7 }
(a)Jump Label in Source
1 function:
2 <function prologue>
3 [...]
4 nop
5 label1:
6 [...]
7 <function epilogue>
8 jump_label:
9 <prepare parameters>
10 call doSomething
11 jmp label1
(b)Jump Label deactivated
1 function:
2 <function prologue>
3 [...]
4 jmp jump_label
5 label1:
6 [...]
7 <function epilogue>
8 jump_label:
9 <prepare parameters>
10 call doSomething
11 jmp label1
(c)Jump Label activated Figure 5.2.: Example of Jump Label implementation in the Linux kernel.
In case theunlikely condition is fulfilled thenopin line 4 is replaced with an unconditionaljump.
R untime Kernel Co de Integrit y
5.2. Kernel Runtime Patching
Load-Time Runtime
Relocation SMP Instructions
External Symbols Jump Labels
Alternative Instructions Function Tracing Paravirtualization Instructions Function Patching
Table 5.1.:Classification of self-patching mechanisms in the Linux kernel
The introduced features are examples in which dynamic runtime patching may take place at any time during execution and not simply at load time, making out-of-band hashing-based troublesome.
Most of the mechanisms described in this section are also relevant for architectures other then x86. All of the runtime patching mechanisms mentioned above can be used at any time within the operation of the kernel. Consequently, the kernel’s code pages are not static, meaning that simple hashing of code pages is not enough to validate its integrity. In addition, as the patching of the kernel’s code depends on the current state of the running system, we must consider semantic information taken from within the guest OS to validate code integrity.
To the best of our knowledge, we are the first to apply semantic knowledge for runtime validation of the kernel.
As we have pointed out in the last section, the Linux kernel applies various code validation of modern kernels is a challenge that requires a deeper understanding of the kernel’s various patching and relocation mechanisms. This means that simple hash-based approaches or approaches that simply make use of the kernel binary are simply not sufficient. In fact, there are several mechanisms for which the integrity and consistency of a change is verifiable but the resulting state still could be used by an attacker. In the following section we will introduce the architecture of our proposed solution to validate the kernel code during runtime.
5. Runtime Kernel Code Integrity
In the simple case, with alternative instructions, an invalid change might only be an instruction which is normally not used on the specific architecture. However, in the case of jump labels, an attacker might enable a hook within the system, while the systems internal state is not aware of the hook.