Department of Computer Science and Engineering
Helsinki University of Technology
This paper introduces concepts of the security kernels as well as two examples of them: Kernelized Security Operating System and Honeywell Secure Communications Processor. The security kernel is a methodology that provides the functionality of the operating system and good internal security in multiuser systems. Especially, they are useful in organizations where different users are trusted on different levels. Also, security kernels are able to co-operate over networks which is very important nowadays.
Keywords: kernel, security, policy, KSOS, Scomp
Even nowadays individual computer system protection is based quite much on physical protection such as locks and guards. Communication between different systems can be protected well cryptographically. However, many systems do not offer proper security functions inside the system. Thus, if someone has access into the system he or she has good changes to access information where he or she is not authorized to access. Also, in many organizations, there is need to trust some people more than others.
One way to meet those needs is to use security kernel approach. The concept of security kernel provides principles which is applicable to all types of systems, from general purpose, multiuser systems to special purpose systems. All access requests are checked by the kernel and it should be impossible to access data where the user is not authorized. Multilevel access rights can be provided by setting access rights individually to every person (or subject).
This paper introduces concepts of security kernels. Chapter 2 represents general principles of security kernels. After the principles of the security kernels are introduced two different kind of implementations are introduced shortly. Chapter 3 introduces Kernelized Secure Operating System and Chapter 4 Honeywell Secure Communications Processor. Chapter 5 introduces security kernels in networking environment because networks are extremely important currently.
2. Principles of Security Kernels
The security kernels can be divided into two categories: actual security kernels and trusted computing bases (TCB). In  the security kernel is defined as an isolated portion of a computer system that is designed to enforce the security policy of the system. A TCB is defined as the totality of hardware and software protection mechanisms responsible for enforcing the security policy of a given system. The difference is small and often security kernels and TCBs are viewed as synonymous. However, the (small) difference is that security kernels involve an isolated portion of a system architecture for security functions but in TCBs security functions may be spread throughout various portions of a system. These are discussed in detail later on this chapter.
The security kernels cannot guarantee full protection. They are as efficient as the chosen policy which is discussed in Section 2.2. Additionally, they cannot protect system from authorized, careless user. For example, users select quite often easily guessed passwords or write them down which makes intruders work relatively easy.
2.1 Reference Monitor
In the security kernel approach very important concept is the reference monitor which is an abstract notation adopted from the models of Butler Lampson. The reference monitor provides an underlying security theory for conceptualizing the idea of protection. In a reference monitor all active entities such as people or computer processes reference to passive entities such as documents or segments of memory using a set of correct access authorizations. Every reference to passive entities or change of authorizations must go trough the reference monitor. The access control information is stored into a database and important security events are stored into the audit file. The idea of reference monitor is illustrated in Figure 2-1.,
2.2 Kernel Security Policies
The policy provided by the reference monitor a fundamental thing. All systems are "secure" only with the respect to a specific policy. A well formed policy identifies all the permissible model of access between the subjects (active entities) and the objects (passive entities). The external policy that corresponds to the people, paper, and methods of accessing information in the real world must be interpreted in a way that allows the policy to apply to the internal entities of the computer system. This requirement for precisely defined policy is a primary distinction between a security kernel based system and several other efforts to develop security relevant operating systems, such as "capability" machines. These other systems tend to strive for general purpose protection, yet do not have any definitive criteria for what is security relevant. The mechanisms in these systems essentially provide a computer with special security features. On the contrary, the security kernel approach explicitly addresses both policy and mechanism. If general purpose protection mechanisms are well-defined and augmented to enforce a specific policy, they can provide underlying base for subsequent security kernel construction.
Two formally defined policies are necessary: non-discretionary and discretionary. All mandatory security rules that are valid for all the users are included in non-discretionary policy. On the other hand, a discretionary policy contains rules that can be specified for each user individually. The protection policy used by the security kernel is a set of mathematical rules that constructs a formal security model. The rules of the model have to consider both types of policies.
There are two key issues that are needed to determine whether the model of a policy is sufficient. Firstly, the model must define the information protection behavior of the system as a whole. Merely modeling distinct operations with respect to individual assertions about a protection mechanism does not indicate much about overall system security and can be misleading. Secondly, the model must include "security theorem" to ensure that the behavior defined by the model always complies with the security requirements of the applicable policy. The most security kernels are based on the Bell-LaPadula model which provides rules for preventing unauthorized observation and modification of information. By representing the security kernel as a finite state machine, these rules define allowable transmissions from one secure state to next. For further information about these models refer to  or .
Within the model each subject and object of the reference monitor is given a security identifier termed an access class. The access classes are used in each state transmission to check if the subject is allowed to access the object. The access classes can be organized to a mathematical structure called lattice. It defines the relations among access classes allowing the reference monitor to determine one access class is less than, greater than, equal to, or disjoint another. Examples of environments in which access classes from a lattice include privacy protection compartments of medical data, financial data, criminal records, and the hierarchical government security classification.
2.3 Security Kernel
Figure 2-2 illustrates general purpose operating system with on-line, interactive users. The kernel provides a relatively small and simple subset of operating system functions. The kernel primitives are the interface of this subset to the rest of the operating system (supervisor mode). The supervisor primitives provide the general-purpose operating system functions used by the applications.
Usually, an operating system consists of several functional areas such as process management, I/O control, and file system management. Some of the functions are security relevant and they must be placed into the kernel. The rules of policy model, discussed in Section 2.2, help to identify security relevant functions. Some of the parts of the operating system must be in the kernel because the model requires that these resources are virtual that their location can be hidden from untrusted software. The functions that provide useful common utilities do not manage anything shared among users and those that address denial of service are outside the scope of the security policy and can generally be in the supervisor.
Often systems require a security policy that is more specifically tailored to their needs than that defined by the basic security model. This tailored policy is generally exercised on a limited basis for infrequent operations and may apply only under special circumstances or to a special class of users. If this extented policy is implemented into the kernel, usually a set of interfaces that can be invoked by only certain trusted subjects is provided. Trusted subjects have some internal identifier, e.g., a privilege indicator. When a running program has such privileges, it may be able to perform actions not permitted by the access checks built into normal kernel functions.
Trusted subjects may be needed to perform system maintenance such as access policy controlling for untrusted subjects. Sometimes normal users invoke certain trusted subjects to perform security sensitive functions. For example, since the basic security model does not allow an untrusted subject to lower the access class of information, the occasional need for downgrading a segment that a user accidentally over-classifies is satisfied by providing a trusted subject for the user. Trusted subjects are often implemented as asynchronous processes, called trusted processes, or as extensions of the kernel itself, called trusted functions. Regardless of the implementation technique, trusted subjects must adhere to the same engineering principles as the kernel if the security policy is to be correctly implemented. Other than the implementation technique, the only difference is the specific security policy enforced.
The security kernel is organized and integrated into the system in system dependent way. However, in a typical computer systems relying on security kernel the security kernel organization resembles the organization presented in Figure 2-3.
The most obvious differences between a security kernel and an operating system are quantitative and follow from the high priority given to the security in comparison to flexibility, functions, performance, cost of development, ease of use, and other factors that are more important to operating system. The most direct quantitative impact is caused by the requirement of kernel verifiability (refer to Section 2.5"), which dictates that the kernel have a primitive interface and be much simpler and smaller than a full operating system, while at the same time insisting that it be closely scrutinized.
The security kernel can be implemented with a new operating system or it can be implemented into an existing system. It is not always reasonable to build a new operating system because it is fairly difficult and lots of applications exist for the old systems. There are three basic approaches to build operating systems with security kernels (refer to Figure 2-4):
2.4 Trusted Computing Base
The concept of the TCB was originally introduced by the U.S. Department of Defense National Computer Security Center to support security evaluations. The word "trusted" in the TCB denotes that TCBs are highly secure. As stated earlier in TCBs the security functions may be spread throughout various portions of the system in contrary to the security kernels. Several methods have been identified for enforcing the security policy of the system. For example, firewalls can be build between different portions of a computer system.
As with security kernels, minimization of the TCB part of the system is desirable. The smaller the trusted part the easier is the task to prove it to work correctly. For example, for large systems formal verification is usually extremely difficult or even impossible.
TCB can be layered. In a layered system various layers of functionality are created and organized into a hierarchy that is represented by a diagram as in Figure 2-5. In the diagram lower layers offer services to higher layers. That is, each layer is trusted to provide the service required by its lower layer. Often, the functional distinction between the layers of the TCB is small. Instead, the trustworthiness and assurance that is assumed and associated with the various layers of TCB is critical concept. Therefore, degrees of trustworthiness might be desired as a means for associating certain layers of TCB with the proper assurance.
2.5 Goals of Kernel Design
The security kernel and the reference monitor must satisfy the following three fundamental principles :
The principle of completeness requires that a subject does not reference an object without invoking the reference monitor and all access requests to information flows trough the kernel. Although this may seem to be easy to implement, there are several important differences between the demands made by this principle and the way that operating systems are generally implemented.
In operating systems the information is usually located in obvious places such as files, memory, and I/O buffers. The operating system attempts to control access to these objects. However, the completeness principle is not fulfilled with an "ad hoc" definition of the objects because any piece of information is an object despite of its size or intended use. For example, information storages are file and directory names, status register, and dynamic data maintained by the operating system (such as current users, processes, resource allocation). To fulfill the completeness principle the access policy to these objects must be defined and enforced by the reference monitor.
The completeness principle places requirements to hardware, as well. If untrusted software is allowed to run without checking each instruction, the hardware must ensure that access controls are not bypassed specified by the kernel. For example, all references to memory, registers, and I/O buffers must be checked.
The principle of isolation requires that the kernel must be tamper-proof. The practical implementation of this principle requires co-operation between the software and hardware. The hardware must offer a way to the kernel to prevent user programs from accessing to the kernel code or data. That is the same memory management mechanism that the kernel uses to prevent processes from accessing each others data. Also, user programs must be prohibited to execute privileged memory management instructions used by the kernel.
If the hardware of the system supports previously described features it is difficult to perform successful direct attack on the kernel by writing the kernel's code or memory area, or executing privileged instructions. More isolation could be achieved by running kernel code in read-only memory. However, direct write is difficult to implement successfully. A much better attack would be try to get the system use attacker's programs in privileged mode thereby offering control to the system without touching the kernel code or data.
The principle of verifiability requires that the kernel designer can convince others that the kernel works correct. The methods which can be used for this are:
Code inspection and testing are the most used verification techniques like in other software projects. However, these methods are not necessarily very efficient. The primary technique that supports the verifiability argument for a security kernel is development of a mathematical model of security. Security is defined precisely by the model and the functions in the model are formally proved to be consistent with this definition. The model must be structured in a way that lends itself to an argument that the kernel implementation adheres to the model.
In order to make verifiability possible the design of the kernel should fulfill "good" software design strategies such as structured design, modularity, information hiding, layering, abstract specification, simplicity of the kernel interface, and minimization of the size of the kernel.
3. Case: Kernelized Secure Operating System
This chapter gives an overview to Kernelized Secure Operating System (KSOS) formerly known as Secure Unix. KSOS is developed by Department of Defense of the US. This chapter is based on  unless otherwise explicitly stated.
KSOS is intended to provide a provably secure operating system for larger minicomputers. KSOS is able to handle information at various security levels. The goal of the system is to provide strong assurances that it is impossible for an unauthorized user to have access to the information.
KSOS is very much similar to the Unix operating system. The only changes are tightened security checking of some of the operating system calls and several new calls. All existing Unix applications run without recompilation on KSOS providing they do not violate the security model of the system.
The structure of KSOS is illustrated in Figure 3-1. It consists of security kernel, Unix emulator, and trusted and untrusted non-kernel security related software (NKSR).
The functions of the security kernel is to provide a simple operating system which can be shown to be secure. The kernel centralizes the control of all the resources in the system. It mediates each access attempt by a user process and only permits those accesses which comply with the access control policy. The kernel resides in the most privileged address space of the machine (kernel mode) where it has access to all of the raw hardware and memory management facilities.
The Unix Emulator is a logical part of each Unix process which resides in the supervisor mode address space of the process. It maps the user's Unix system calls into the corresponding kernel calls.
The non-kernel system software is a collection of autonomous processes performing support services for the system. Contrary to Unix, KSOS does not have services like login embedded in the operating system. Those services are performed by trusted processes which are placed outside of the kernel. They are like any other process but they have privilege to violate selectively the security model of the kernel. The Unix emulator is considered as untrusted and is not intended to be verified. Therefore, trusted software must use the kernel directly.
KSOS is verified by using the Special formal specification language and the Hierarchical Development Methodology. Additional assurances were provided via Unix configuration management and controls, well-defined design and code reviews, and extensive security and functional testing of the KSOS security kernel.
KSOS has few problems. Certain Unix features (e.g., process family sharing of open-file seek pointers) are incompatible with the requirements of secure systems. Additionally, the operating system emulator is much slower than the emulated system. Reasons to this performance reduction are, e.g., the incompatibility of the security kernel with the emulated system, the hardware capabilities, and the code generated by the language compiler.
4. Case: Honeywell Secure Communications Processor
This chapter gives an overview to the Honeywell Secure Communications Processor (Scomp). Scomp has its origins in a joint Honeywell - Air Force program called Project Guardian. The purpose of that project was to enhance the security of the Multics system developed by Honeywell. This chapter is based on  unless otherwise explicitly mentioned.
Scomp supports a variety of specialized applications that require the processing of information with multilevel security attributes. The Scomp system is an implementation of a hardware/software general purpose operating system based on the TCB concept.
Scomp hardware supports Multics like, hardware enforced ring mechanism, virtual memory, virtual I/O processing, page-fault recovery support, and performance mechanisms to aid in the implementation of an efficient operating system. The Scomp trusted operating program (STOP) is a security kernel based, general operating system that provides a multilevel hierarchical file system, interprocess communication, security administrator functions, and operator commands.
Originally, the plan was to develop a security kernel and an emulator for the existing operating system. Also, the plan was to support Unix interface in such way that Unix applications could be used in Scomp. However, due to problems of KSOS, presented in Chapter 3, this plan was abandoned. Instead of providing a full Unix compatibility, Honeywell took a new approach to build a new interface for Scomp called Scomp kernel interface package (SKIP). SKIP does not try to emulate a specific system but it takes advantage of the underlying hardware and security kernel architecture to provide an efficient interface for applications.
The Scomp system satisfies the requirements of the reference monitor by providing complete meditation, isolation, and verifiability. Mediation of provided trough the interaction of the Scomp hardware and software (refer to Figure 4-1). The software provides the initial meditation of a request for access using the internal data structures of the kernel. It validates the request for both the subject and object of the requested operation (e.g., read or write). The software builds a data element, in the form of a descriptor, for use by the hardware. The descriptor consists of access permission and the location of the object.
Isolation is provided by the hardware implementation of a Multics-like four ring mechanism with ring 0 containing the security kernel. This implementation, developed from the Multics architecture, includes controlled ring-crossing to allow less privileged software to access an inner ring for a service function.
In Scomp the security relevant software is verified using two methods: the SRI International Hierarchical Development Technology and Gypsy methodology. These methods are outside of the scope of this paper.
5. Networking and Security Kernels
Nowadays networking and number of distributed systems are growing rapidly. As stated earlier, the security kernel might be very useful in large organizations such as hospitals, government, and large corporations. Therefore, it is important that systems using security kernels can utilize networks and preferably, maintain access classes between different systems. This chapter describes some basic ideas how security kernels can be used with networks.
If the distributed system consists of two (or more) security kernel based operating systems and the same policy is wanted to use on the distributed system as within kernels, the individual kernels must co-operate in some manner. There are two basic ways to organize co-operation.
One way (illustrated in Figure 5-1a) to do this is to allow the kernels to communicate directly with one another, exchanging control information as needed to co-ordinate the exchange of traffic. This technique requires a set of kernel protocols, i.e., trusted kernel-to-kernel protocols. Naturally, the communication path between kernels must be protected. The kernel protocols have very difficult synchronization requirements that must await solutions to still-open research issues.
Another way, which is much easier to implement, is illustrated in Figure 5-1b. The kernels are kept as disjoint as possible, relying on each kernel to enforce subject-to-object access within its own computer system and minimizing the amount of trusted control information that must be exchanged between kernels. Actually, for a multilevel security policy, the only trusted control information that needs to be exchanged is the access class of the network objects. The transmitting kernel inserts an access class label on a message based on the access class of the subject that created the message. The receiving kernel uses that access class to determine who on its own system may read the message. The kernels must still to trust each other to insert and reject the access class labels according to their common policy. Trusted labeling can be provided in various ways without trusting all of the network software that implements used protocols, but in general adding trusted labeling with the same degree of assurance as is possessed by the kernel cause trusting a considerable amount of network software. If the connection is protected by the transport layer, the label is required only when the virtual circuit is established, and not on each message that is changed over the circuit.
When security kernel communicates with an untrusted system (Figure 5-1c), the security kernel must treat the non-security kernel as a single subject that is unable to enforce or provide any reliable labeling. With a multilevel security policy, this means that the untrusted system may communicate with subjects on the kernelized system at only a single access class, which the kernel determines by authentication the remote system and knowing the correct access class from internal tables.
This paper considered the concepts of the security kernels. The security kernel is a methodology that provides the functionality of the operating system and internal security in the system. The kernel provides security controls that are effective against most internal attacks. The concept of reference monitor requires that the system checks every access to objects and decides if the subject is authorized for accessing based on some pre-defined policy.
The security functions can be placed to an isolated portion of the computer system which is the actual security kernel approach (e.g., KSOS). Another way is to spread security functions throughout various portions of the system. This approach is called trusted computing base (e.g., Scomp). There are three strategies that can be used when designing a security kernel (or trusted computing base). Two of them was introduced using real implementations. The compatible system with an existing operating system can be build by designing a security kernel and emulator for the existing system (KSOS). The second way is to build a new operating system with security kernel (Scomp). The third way is to build new kernel which support existing operating system without changes.
The goals of the security kernel are the following three principles: completeness, isolation, and verifiability. The security kernel should fulfill these requirements.
Today networking is growing enormously and security kernels must be capable to operate with other security kernels and untrusted system. Communication with other security kernels can be arranged by using specialized kernel protocol or relying each system to perform check independently. The latter is preferable. Communication with untrusted system can be arranged treating untrusted system as a subject.