Dealing with Disaster: Surviving Misbehaved kernel Extensions
Margo Seltzer, Yasuhiro Endo, Christopher Small and Keith Smith
Today's extensible operating systems allow applications to modify
kernel behavior by providing mechanisms for application code to run
in the kernel address space. The advantage of this approach is that
it provides improved application flexibility and performance; the
disadvantage is that buggy or malicious code can jeopardize the
integrity of the kernel. It has been demonstrated that it is feasible
to use safe languages, software fault isolation, or virtual memory
protection to safeguard the main kernel. However, such protection
mechanisms do not address the full range of problems, such as
resource hoarding, that can arise when application code is introduced
into the kernel.
In this paper, we present an analysis of extension mechanisms in the VINO kernel. VINO uses software fault isolation as its safety mechanism and a lightweight transaction system to cope with resource-hoarding. We explain how these two mechanisms are sufficient to protect against a large class of errant or malicious extensions, and we quantify the overhead that this protection introduces.
We find that while the overhead of these techniques is high relative to the cost of the extensions themselves, it is low relative to the benefits that extensibility brings.