CVE-2006-5755

Step 1: Trigger SYSENTER: A local, unprivileged user process executes the SYSENTER instruction, potentially setting the NT flag in the EFLAGS register. This can be achieved through crafted system calls or by exploiting other vulnerabilities to gain control of the execution flow. Step 2: Context Switch: The kernel performs a context switch, moving from the malicious process to another process. This switch should save and restore the EFLAGS register. Step 3: Incorrect EFLAGS Restoration: The kernel incorrectly restores the EFLAGS register for the new process. Because the NT flag was set by the previous process, it is now incorrectly restored for the new process. Step 4: IRET Execution: The new process attempts to return from an interrupt or system call using the IRET instruction. Because the NT flag is set, the IRET instruction attempts to perform an invalid operation. Step 5: Kernel Panic: The invalid operation triggered by IRET causes a kernel panic, resulting in a system crash and denial of service.

The vulnerability lies in the x86_64 architecture's context switching mechanism within the Linux kernel. Specifically, the kernel fails to correctly save and restore the EFLAGS register, which contains crucial processor flags. The flaw occurs when a malicious process leverages the SYSENTER instruction to set the NT (Nested Task) flag within EFLAGS. During the subsequent context switch to another task, the kernel's IRET (Interrupt Return) instruction attempts to restore the incorrect EFLAGS value, including the NT flag. This leads to an invalid state, triggering a kernel panic and system crash. The root cause is a missing or incorrect handling of the EFLAGS register's state during the context switch process, specifically the failure to preserve and restore the NT flag's state correctly.