Chapter 9. Building and Installing a Driver
Prev | Part III. Kernel-Level Drivers | Next

## Chapter 9. Building and Installing a Driver

After a kernel-level driver has been designed and coded, it must be compiled, linked, and installed. The topics in this chapter describe the major steps of this process, as follows:

- “Defining Device Numbers” covers the choice of major and minor device numbers.

- “Defining Device Special Files” describes options for creating the file or files controlled by the driver.

- “Compiling and Linking” covers the compiler and linker options used for driver modules.

- “Configuring a Nonloadable Driver” describes the configuration files used to set up a driver loaded at boot time.

- “Configuring a Loadable Driver” describes the additional configuration needed for a loadable driver.

## Defining Device Numbers

The topics “Major Device Number” in Chapter 2 and “Minor Device Number” in Chapter 2 cover the purpose and use of the device numbers. Major and minor numbers were once very important in the device driver design because they were the primary input that distinguished a device to a device driver upper-half entry point. In current IRIX, this is only the case for legacy drivers in older machines. Contemporary drivers take their input from a vertex of the `hwgraph` (see “Hardware Graph” in Chapter 2).

The historical use of device numbers can be summarized as follows:

- Both numbers are encoded in the inode of a device special file in `/dev`.

- The major number selects the device driver.

- The minor number specifies the logical unit, and can encode device features.

- Both numbers are passed as a parameter to driver entry points.

Part of creating and installing a device driver is the selection of device numbers and the definition of device special files.

### Selecting a Major Number

If your driver does not use the hwgraph, you must select a major number to stand for your driver. The numbers that already exist are listed in `sys/major.h`. However, the major number should not be coded into the driver. Typically the driver code does not need to know its major number, and if it does, the driver should discover its major number dynamically. A method of doing this is discussed under “Variables Section”.

A driver is associated with its major number in the `master.d` configuration file. When the driver discovers this number dynamically, the system administrator is free to change major numbers in `/var/sysgen/master.d` files to correct conflicts between one product and another.

### Selecting Minor Numbers

When a driver is called to service a device special file defined only in `/dev`, it receives a device minor number comprising 18 bits of information. You design the content of these numbers to give your driver the information it needs about each device special file. Typically you encode a unique device unit number so the driver can index device information from an array. (When the hwgraph is used, a pointer to the device information is stored in the hwgraph vertex instead.)

Examine the `/dev/MAKEDEV` script to see some techniques for assembling minor numbers dynamically based on the hardware inventory and other commands.

## Defining Device Special Files

As described under “Device Special Files” in Chapter 2, the association between a device and a driver is established when a process opens a device special file in the `/hw` or `/dev` directory. Without at least one device special file, a device can never be opened.

### Static Definition of Device Special Files

The system administrator can create device special files using `mknod` or `install` (see “Making Conventional Device Files” in Chapter 2). This can be done manually, or through an installation script, or as an exit operation of the software manager program. The device special files can be created at any time—whether or not the device driver is installed, and whether or not the hardware exists. The special device files continue to exist until the administrator removes them.

### Dynamic Definition of Device Special Files

A more sophisticated approach is to have the device special files created, or recreated, dynamically each time the system boots. This was the purpose for which `/dev/MAKEDEV` (see “The Script MAKEDEV” in Chapter 2) was introduced—it removes and redefines device special files based on information in the hardware inventory. In current IRIX, all entries in the `/hw` filesystem are created dynamically by device drivers as devices are attached.

#### Definition and Use of /hw Entries

The kernel creates the upper levels of the hardware graph to represent addressable units of hardware in the basic system—modules, buses, and slots. While probing buses, it finds devices, and calls upon device drivers to attach them (see “Entry Point attach()” in Chapter 7 and “Entry Point edtinit()” in Chapter 7). At these times, the driver has the responsibility of extending the hwgraph with vertexes that provide access to the device (see “Extending the hwgraph” in Chapter 8).

Because hwgraph entries are always created dynamically, and can be created and destroyed while the system is running, the initial set of pathnames in `/hw` are not stable and should not be written into user scripts and source code. Your driver can create additional vertexes in the hwgraph (see “Extending the hwgraph” in Chapter 8), both when attaching a device and later, when `ioconfig` runs (see “Using ioconfig for Global Controller Numbers” in Chapter 2).

## Compiling and Linking

You compile a kernel device driver to an ELF binary using shared libraries. The compile options differ between 32-bit and 64-bit modules.

### Platform Support

If you are building a device driver that you wish to use on multiple platforms, you should build a different driver for each CPU board type (for example, IP22) that you want to run it on. You can use the `hinv` command to determine the host architecture (see `hinv` (1M)) and then specify the board type in the `Makefile` as described in the next section.

### Using /var/sysgen/Makefile.kernio

The file `/var/sysgen/Makefile.kernio` is a sample Makefile for compiling kernel modules. You can include it from a Makefile of your own to set the definitions of compiler variables and compiler options appropriately for different CPUs and module types.

The `Makefile.kernio` file tests the following environment variables, which you set:

CPUBOARD | Set to the type of CPU used in the target system, for example IP19, IP22, IP27 (see the `sys/cpu.h` header file).
COMPILATION_MODEL | Set to 64 for a 64-bit kernel module, or to 32 for a 32-bit kernel module.

The purpose of the rules in `Makefile.kernio` is to set numerous compiler variables appropriately for the CPU type and execution model. It also sets compiler options into a Make variable CFLAGS. Owing to the number of compiler variables and the importance of getting them right for each CPU type, Silicon Graphics strongly recommends that you invoke `Makefile.kernio` from your own makefile.

| Note: `Makefile.kernio` is designed for nonloadable drivers. In particular it sets the compiler option `-G8`, which is valid for nonloadable drivers. For loadable drivers, use the file `/var/sysgen/Makefile.kernloadio` as a sample Makefile. This sets the `-G0` flag and other options appropriate for loadable drivers.

### Compiler Variables

The compiler variables listed in Table 9-1 are tested in system header files. They are usually defined on the compiler command line. The rules in `Makefile.kernio` set definitions of these variables appropriately for different CPU types.

Table 9-1. Compiler Variables Tested by System Header Files

Variable | Meaning
--- | ---
_KERNEL | Compile for a kernel module, not a user program.
MP | Compile for a multiprocessor.
_MP_NETLOCKS | Compile network driver (only) for multiprocessor TCP/IP.
STATIC=static | Use of pseudo-declarator STATIC is converted to real static.
_PAGESZ=16384 | Compile for a kernel using 16K memory pages.
_PAGESZ=4096 | Compile for a kernel using 4K memory pages.
_MIPS3_ADDRSPACE | Kernel for a MIPS3 machine.
R10000 | Target machine is the R10000.
TFP | Target machine is the R8000.
R4000 | Target machine is the R4000.
IPnn | Target machine uses the IPnn CPU module, one of IP19, IP20, IP21, IP22, IP25, IP26, IP27, IP28, IP30, and IP35 are currently supported.
EVEREST | Compile for a Challenge or Onyx system.
BADVA_WAR, JUMP_WAR, PROBE_WAR | Compile workaround code for bugs in certain R4x00 revisions.
_IP26_SYNC_WAR, _NO_UNCCHED_MEM_WAR | Compile workaround code for IP26 bugs.
R10000_SPECULATION_WAR | Compile workaround code for bug in certain R10000 revisions.
USE_PCI_PIO | Compile workaround for IP32 PIO bug (see `sys/PCI/pciio.h`).

### Compiler Options

Some of the `cc` and `ld` options needed to compile and link a kernel-level driver are shown in Table 9-2. The complete and most current set is defined in `Makefile.kernio`.

Table 9-2. Compiler Options Kernel Modules

Option | Purpose
--- | ---
`-non_shared` | Do not compile for shared libraries (no dynamic linking).
`-elf` | Compile and link an ELF binary.
`-64` | Set for any kernel using the 64-bit execution model. 32-bit kernel does not set any specific flag.
`-mips4` , `-mips2` | Select the MIPS4 instruction set only for the R10000 CPU. Use MIPS2 for others.
`-G 8` | In a nonloadable driver, use the global table for objects up to 8 bytes.
`-G 0` | In a loadable driver, do not use the global table. Refer to the `gp_overflow(5)` reference page for a discussion of the global table.
`-r` | Linker to retain symbols—for all drivers (required by loadable drivers, and needed for `lboot`).
`-d` | Force definition of common storage even though `-r` used.
`-Wc,-pic0` | Do not allocate stack space used by shared objects.
`-jalr` | In loadable drivers only, use `jalr` (jump-and-link register) instead of `jal`, whose 26-bit operand may not be enough for subroutine calls from a loaded module to the kernel.
`-TARG:t5_no_spec_stores` | Crucial setting for Indigo2 R10000 only; without it, kernel memory corruption can occur.
`-TENV:kernel` `-TENV:misalignment=1` | Execution environment options for 64-bit compiler.
`-OPT:space` `-OPT:quad_align_branch_targets=F ALSE` `-OPT:quad_align_with_memops=FALS E` `-OPT:unroll_times=0` | Specific optimization constraints for 64-bit compiler.

## Configuring a Nonloadable Driver

When the driver is not loadable, it is linked as part of the IRIX kernel. The following steps are needed to make the driver usable:

1. Place the driver executable file in `/var/sysgen/boot`.

2. Place a descriptive file in `/var/sysgen/master.d`.

3. Place a directive file in `/var/sysgen/system` (or simply add a line to `/var/sysgen/system/irix.sm`).

4. Run `autoconfig` to generate a new kernel.

5. Reboot the system.

Some of these steps are elaborated in the following topics.

### How Names Are Used in Configuration

The process of naming a kernel-level driver begins in a file in `/var/sysgen/system`, such as `/var/sysgen/system/irix.sm`. Names are used as follows:

- A USE, INCLUDE, or VECTOR statement in `/var/sysgen/system` specifies a name, for example

      USE hypothetical

- This statement directs `lboot` to read a file of the same name in `/var/sysgen/master.d`. In this example, the file would be `/var/sysgen/master.d/hypothetical`.

- The file in `/var/sysgen/master.d` specifies the prefix for driver entry points, for example hypo_.

- The same name with the suffix `.o`, is searched for in `/var/sysgen/boot`—in this example, `/var/sysgen/boot/hypothetical.o`. This object file is linked with the kernel.

- The public symbols in the object file are searched for names that start with the prefix, for example `hypo_attach()`. These are noted in the kernel switch table so the driver can be called as needed.

### Placing the Object File in /var/sysgen/boot

The `/var/sysgen/boot` directory, where the kernel object modules reside, is actually a symbolic link to one of the directories `/usr/cpu/sysgen/IP`nn`boot`, where nn is the number of one of the CPU modules supported by the current release of IRIX (see “CPU Modules” in Chapter 1). When you place the object file of a driver in `/var/sysgen/boot`, you actually place it in the CPU directory for the system in use.

### Describing the Driver in /var/sysgen/master.d

You describe your driver in a file with the name of the driver in `/var/sysgen/master.d`. The format of these files is described in two places: the `master(4)` reference page, and in `/var/sysgen/master.d/README`. In addition, you can examine the many examples in the distributed system.

#### Descriptive Line

The first noncomment line of the master file contains a series of fields, delimited by white space, to describe the driver. These fields are listed in Table 9-3.

Table 9-3. Fields of Descriptive Line in Master File

Field Number | Usage | Details
--- | --- | ---
1 | Flags | See Table 9-4 .
2 | Prefix | The string of 1-14 characters that identify the public symbols of driver entry points.
3 | Major number | The major device number found in device special files managed by this driver. When the driver uses the hwgraph, this field contains only a hyphen (-).
4 | Number of sub-devices | Size of the driver's static arrays of device information, or given as a hyphen “-” when the driver stores device information in the hwgraph.
5 | Dependencies | A list of other modules that must be in the kernel for this driver to work—usually omitted except for SCSI drivers.

The important flag values for nonloadable drivers are listed in Table 9-4.

Table 9-4. Flag Values for Nonloadable Drivers

Letter | Meaning
--- | ---
`b` or `c` | Block (b) or character (c) device. One or the other is essential for any device driver.
`f` or `m` | STREAMS driver (f) or STREAMS module (m). Omit for device driver.
`s` | Software driver, either a pseudo-device or a SCSI driver.

The `s` (software-only) flag tells `lboot` not to attempt to probe for hardware. This is the case with software-only (pseudo-device) drivers, and with SCSI drivers. If `lboot` tries to probe for a SCSI device, it fails, and assumes that the device is not present, and does not include your SCSI device driver.

Additional flags (d, r, D, N, R) for loadable drivers are discussed later in the section “Configuring a Loadable Driver”.

#### Listing Dependencies

The descriptive line ends with a comma-separated list of other loadable kernel modules on which this driver depends. The `lboot` command makes sure that it will not load this module if it fails to load a dependency.

In most cases, an OEM driver does not have any dependencies. However, a SCSI driver (see Chapter 16, “SCSI Device Drivers”) should list the name `scsi`, since it depends on the inner SCSI driver. A STREAMS driver might list the name of a STREAMS support module such as `clone` (see “Support for CLONE Drivers” in Chapter 22).

It is possible for you to design a driver in the form of multiple, loadable modules. In that case, you would name your support modules in this field.

#### Stubs Section

Noncomment lines that follow the descriptive line and precede a line beginning “$” are used by library modules—not by device drivers or STREAMS drivers. Each such line specifies an entry point that this module provides, and which is used by the kernel or some other loadable module. These lines instruct `lboot` in how to create a harmless “stub” function in the event that this driver is not included in the kernel—for example, because it is specified by an EXCLUDE line in the system file. The format is discussed in the `master(4)` reference page.

Since a device or STREAMS driver provides only standard entry points that are accessed via the switch tables rather than by linking, drivers do not need to define any stubs.

#### Variables Section

Following the descriptive line (and the stubs section, if any), you can place a line that begins with “$” and, following this, you can write definitions of global variables using C syntax. This text is compiled into a module linked with the kernel. You refer to these variables as extern in the driver source code.

The advantage of defining global variables in the master file is that the initializing expressions for these variables can include values taken from the descriptive line. The following special symbols can be used:

##E | The integer coded as the major number in the descriptive line. The first integer, if a list of major numbers is given.
##C | The number of controllers (bus adapters) of this type.
##D | The number of sub-devices as coded in the fourth field of the descriptive line.

You can use these symbols to compile run-time values for the major device number and the number of supported sub-devices, as specified in the descriptive line of the file, without coding them as constants in the driver. In the source code you can write

    extern major_t myMajNum;
    extern int myDevLimit;

In the master file you can implement the variables using the code in Example 9-1.

Example 9-1. Defining Variables in Master Descriptive File

    $$$
    major_t myMajNum = ##E;
    int myDevLimit = ##C;

(In a loadable driver this technique requires one additional step; see “Master File for Loadable Drivers”.)

### Configuring a Kernel

Once you have placed the binary in `/var/sysgen/boot` and the master file in `/var/sysgen/master.d`, you can configure and generate a new kernel. This is done using the `autoconfig` command, which in turn calls `lboot` to actually create a new bootable file.

The `lboot` program only loads modules that are specified in a file in `/var/sysgen/system`. One command is required to specify the new driver; the command is one of the following:

VECTOR | To specify hardware details, to request a hardware probe at boot time, to load the driver and invoke pfx```edtinit()`.
INCLUDE | To load the driver and invoke pfx```init()`.
USE | To load the driver and invoke pfx```init()` only if the master file exists in `master.d`.

The form of these commands is detailed in the `system(4)` reference page. In addition, you should examine the distributed files in `/var/sysgen/system`, especially `irix.sm`, which contains many comments on the meaning and use of different entries. Specific uses of the VECTOR statement are discussed in the following topics: The form of VECTOR lines for VME devices is discussed under “Configuring VME Devices” in Chapter 12.

You could place the VECTOR, USE, or INCLUDE line for your driver in `irix.sm`. However, since `lboot` treats all files in `/var/sysgen/system` as a single file, you can create a small file unique to your driver. The name of this file is not significant, but a good name is the driver name with the suffix `.sm`.

### Generating a Kernel

The `autoconfig` script invokes `lboot` to read the system files, read the master files, and link all the kernel executables. Provided there are no errors, the output is a new file `/unix.install`. At boot time this file is moved to the name `/unix` and used as the kernel.

During the testing period you may want to keep the original kernel file available as `/unix.original`. A simple way to retain this file is to create a link to it using the `ln` command.

## Configuring a Loadable Driver

You compile and configure a loadable driver very much as you would a nonloadable driver (so you should read “Configuring a Nonloadable Driver” before reading this section). The differences are as follows:

- You provide an additional global variable with the public name pfx `mversion`.

- You use a few different compile options.

- You decide when the driver should be loaded, and use appropriate flags in the descriptive line in the master file.

For more background on loadable modules, see the `mload(4)` and `ml(1)` reference pages.

| Note: You may not call `sthread_create()` in a loadable driver, because the stack must be in direct mapped (K0) space. The `sthreads` facility has been superseded by `pthreads(5)` .

### Public Global Variables

To be loadable, a driver must specify a pfx `devflag` entry point containing the D_MP or D_MT flag (see “Driver Flag Constant” in Chapter 7).

Any loadable module must define a public name pfx `mversion`, declared as follows:

    #include <sys/mload.h>
    char *pfxmversion = M_VERSION;

Note the exact spelling of the variable; it is easy to overlook the letter “m” after the prefix. If the variable does not exist or is incorrectly spelled, an attempt to load the driver will fail.

### Compile Options for Loadable Drivers

Use the `-G 0` option when compiling and linking a loadable driver, since the global option table is not available to a loadable driver. You must also use the `-jalr` option in a loadable driver (see “Compiler Options”).

In a loadable driver, link using the `-r` and `-d` options to retain the symbol table yet generate a bss segment.

### Master File for Loadable Drivers

The file in `/var/sysgen/master.d` for a loadable driver has different flags.

In the flags field of the descriptive line of the master file (see “Descriptive Line”), you specify that the driver is loadable, and when it should be loaded. The possible flags are listed in Table 9-5.

Table 9-5. Flag Values for Loadable Drivers

Letter | Meaning
--- | ---
`b` or `c` | Block (b) or character (c) device. One or the other is essential for any device driver.
`f` or `m` | STREAMS driver (f) or STREAMS module (m). Omit for device driver.
`s` | Software driver, either a pseudo-device or a SCSI driver.
`d` | Specifies that this is a loadable driver.
`R` | Auto-register the module (discussed in text).
`D` | Load, then unload, at boot time, in order to let the driver initialize the hardware inventory.
`N` | Prevent this module from being automatically unloaded even when it has a pfx```unload()` entry point.

When the `d` flag is given for an included module, `lboot` parses the master file for the driver. Global variables defined in the variables section of the master file (see “Variables Section”) are defined and included in the kernel. However, object code of the driver is not included in the kernel, and the names of its entry points are not entered into the kernel switch table.

Such a driver has to be manually loaded with the `ml` or `lboot` command before it can be used; and it cannot be used from the miniroot.

### Loading

A loadable driver can be loaded by the `lboot` command at boot time, and by the `ml` command while the system is running. The following steps occur when a driver is loaded:

1. The object file header is read.

2. Memory is allocated for the driver's text, data, and bss segments.

3. The driver's text and data are read.

4. The text and data are relocated. References to kernel names and to global variables named in the master file are resolved.

5. Entry points are noted in the appropriate kernel switch table.

6. The pfx `init()` entry point is called if one is defined.

7. If the driver is named in a VECTOR statement and has a pfx `edtinit()` entry point, that entry point is called for each VECTOR statement that names the driver.

8. The pfx `start()` entry point, if any, is called.

9. The pfx `reg()` entry point, if any, is called.

Space allocated for the module's text, data, and bss is located in node 0 of an Origin2000 system. Static buffers in loadable modules are not necessarily physically contiguous in memory.

A variety of errors can occur when a module is loaded. See the `mload(4)` reference page for a list of possible causes.

#### Effect of `D' Flag

Normally a loadable driver is not loaded at boot time. It must be loaded sometime after boot using the `ml` command. When the `D` flag is included in the descriptive line in the descriptive file, `lboot` loads the driver at boot time, and immediately after calling pfx `start()`, unloads the driver. This permits the driver to test the hardware and set up the hwgraph and hardware inventory.

### Registration

A loadable module is “registered” by loading it, then placing a stub entry in the pfx `open()` and pfx `attach()` column of its row of the switch table, and unloading it again. The stub entry points are invoked when the driver is needed, and the code of the entry points initiates a load of the driver.

Registration of this kind can be done automatically during bootstrap, or later using the `ml` command. Once it has been registered, a driver is loaded automatically the first time the kernel needs to attach a device supported by this driver, or the first time a process attempts to open a device special file managed by this driver. You can also load a registered driver in advance of any use with the `ml` command—loading implies registration.

| Note: Try not to confuse this “registration” with a driver's registration with the kernel to handle a particular type of device.

Registration is done automatically for each master descriptive file that contains the `d` (loadable) and `R` (register) flags. Autoregistration is done at bootstrap phase 2. It is initiated by the script `/etc/rc2/S23autoconfig`. Registration can be initiated manually at any time after bootstrap by using the `ml` or `lboot` command with the `reg` option (see the `ml(1M)` and `lboot(1M)` reference pages).

#### Reloading

When a registered driver is reloaded, the sequence of events listed under “Loading” occurs again. There is one exception: the pfx `reg()` entry point is not called when a registered driver is reloaded from a stub. (The complete sequence occurs when an unregistered driver is explicitly loaded by the `ml` command.)

### Unloading

A module can be unloaded only when it provides an pfx `unload()` entry point (see “Entry Point unload()” in Chapter 7). The `N` flag can be specified in the master file to prevent automatic unloading in any case.

A loaded module is automatically unloaded following a delay after the last close of a device it supports. The delay is configurable using `systune`, as the module_unld_delay variable (see the `systune(1)` reference page). You can use `ml` to specify an unloading delay for a particular module.

The `lboot` or `ml` command can be used to unload a module before the delay expires, or to manually override the `N` flag.

The unload sequence is as follows:

1. The kernel verifies that all opens of the driver's devices have been closed. The driver cannot be unloaded if it has open devices or active mmaps.

2. The pfx `unreg()` entry point is called, if one exists. This gives the driver a chance to unregister as a provider of service for a particular device type. If pfx `unreg()` returns nonzero, the process stops.

3. The pfx `unload()` entry point is called. If it returns nonzero, the process stops.

4. The module is removed from memory. If it had been registered (R flag), stubs are again placed in the pfx `open()` and pfx `attach()` column of its row of the switch table.

Experience has shown that most of the problems with loadable drivers arise from unloading and reloading. The precautions to take are described under “Entry Point unload()” in Chapter 7.

Prev | Table of Contents | Next
Chapter 8. Device Driver/Kernel Interface |  | Chapter 10. Testing and Debugging a Driver