Document revision date: 30 March 2001

## OpenVMS I/O User's Reference Manual

Previous | Contents | Index

### 10.2.2 Using Fast Path

This section describes how to manage Fast Path.

#### 10.2.2.1 Fast Path System Parameters

There are three FAST_PATH system parameters:

- FAST_PATH
- FAST_PATH_PORTS
- IO_PREFER_CPUS

These parameters can be used to control Fast Path as follows:

> #### FAST_PATH

> FAST_PATH is a static system parameter that enables (1) or disables (0) the Fast Path performance features for all Fast Path-capable ports.

> Starting in OpenVMS Version 7.2, Fast Path is enabled by default. In Versions 7.0 and 7.1, Fast Path was disabled by default.

> #### FAST_PATH_PORTS

> FAST_PATH_PORTS is a 32-bit mask. Once Fast Path has been enabled by setting FAST_PATH to 1, FAST_PATH_PORTS can be used to selectively disable Fast Path for some specific adapter types.

> The value of the FAST_PATH_PORTS system parameter is the sum of the values of the bits that have been set. Following is a description of the bit mask:

> Bit | Mask | Description
> --- | --- | ---
> 0 | 00000001 | 0 = Fast Path is ENABLED for KZPBA ports when FAST_PATH is set to 1.
> |  | 1 = Fast Path is DISABLED for KZPBA ports.
> 1 | 00000002 | 0 = Fast Path is ENABLED for KGPSA ports when FAST_PATH is set to 1.
> |  | 1 = Fast Path is DISABLED for KGPSA ports.

> The remaining bits are reserved for possible future adapter types.

> The default setting for FAST_PATH_PORTS is 0, so neither the KZPBA nor the KGPSA ports are disabled.

> Note that CI drivers are not controlled by FAST_PATH_PORTS. Fast Path for CI is enabled and disabled exclusively by the FAST_PATH system parameter.

> #### IO_PREFER_CPUS

> IO_PREFER_CPUS is a dynamic system parameter that controls the set of CPUs available for use as Fast Path preferred CPUs.

> IO_PREFER_CPUS is a CPU bit mask specifying the CPUs that are allowed to serve as preferred CPUs and thus can be assigned a Fast Path port. CPUs whose bit is set in the IO_PREFER_CPUS bit mask are enabled for Fast Path port assignment. IO_PREFER_CPUS defaults to -1, which specifies that all CPUs are allowed to be assigned Fast Path ports.

> You may want to disable the primary CPU from serving as a preferred CPU by clearing it's bit in IO_PREFER_CPUS. This will reserve the primary for use by non-Fast Path IO operations.

> Changing the value of IO_PREFER_CPUS causes the FASTPATH_SERVER process to execute the automatic assignment algorithm that spreads Fast Path ports evenly among the new set of usable CPUs.

#### 10.2.2.2 Identifying and Setting A Port's Preferred CPU

Following are the full set of commands used to identify and set a preferred CPU for a port.

> #### DCL SHOW DEVICE/FULL or $GETDVI DVI$_PREFERRED_CPU

> To identify the preferred CPU for any Fast Path-capable device when Fast Path is enabled, use the DCL command SHOW DEVICE/FULL to display - whether or not the device supports Fast Path - the current preferred CPU ID and, if set, the User Preferred CPU ID for a port or disk device.

> Alternatively, the $GETDVI system service or the DCL F$GETDVI lexical function will return the preferred CPU for a given device or file. The $GETDVI system service item code is DVI$_PREFERRED_CPU, and the F$GETDVI item code string argument is PREFERRED_CPU. The return argument is a 32-bit CPU bit mask with a bit set indicating the preferred CPU. A return argument containing a bit mask of zero indicates that no preferred CPU exists, either because Fast Path is disabled or the device is not a Fast Path-capable device. The return argument serves as a CPU bit mask input argument to the $PROCESS_AFFINITY system service. The argument can be used to assign an application process to the optimal preferred CPU.

> For an application seeking optimal Fast Path benefits, you can code each application process to identify and run on the preferred CPU where the majority of the process's I/O activity occurs.

> A high-availability feature of OpenVMS Cluster Systems is that dual-pathed devices automatically fail over to a secondary path, if the primary path becomes inoperable. Because a Fast Path device could fail over to another path or port, and thereby, to another preferred CPU, an application should occasionally reissue the $GETDVI in a timer thread to check that process assignment is optimal.

> #### DCL SHOW CPU /FULL

> You can use this DCL command to identify whether a CPU is enabled for use as a preferred CPU, and the current set of ports assigned to that CPU.

> #### DCL SET /PREFERRED_CPU and /NOPREFERRED_CPU

> These commands allow you to specify a CPU or a set of candidate CPUs from which the operating system will choose the CPU to assign to the Fast Path port. The chosen CPU is called the preferred CPU for this Fast Path port. The Fast Path port's interrupt I/O completion processing and I/O initiation processing will be performed on this preferred CPU.

> In addition to selecting the preferred CPU, the User Preferred CPU will be set for this port. Setting the User Preferred CPU prevents the port from being reassigned to another CPU unless the User Preferred CPU is being stopped. The qualifier can be negated. When the /NOPREFERRED_CPUS qualifier is specified, the User Preferred CPU will be cleared for the port, but it still remains a Fast Path port, and the current preferred CPU will not be changed.

> If both /PREFERRED_CPUS and /NOPREFERRED_CPUS are specified on the same command line, /NOPREFERRED_CPUS is ignored.

> #### $QIO IO$_SETPRFPATH ! IO$M_PREFERRED_CPU [!IO$M_SYS_ASSIGNABLE]

> You can change the assignment of a Fast Path port to a CPU by issuing a $QIO IO$_SETPRFPATH (Set Preferred Path) to the port device, for example, PNA0. The IO$M_PREFERRED_CPU modifier must be set, and the $QIO argument P1 must be set to either zero or the address of a 32-bit CPU bit mask with a bit set indicating the new preferred CPU. On return from the I/O, the port and its associated devices are all assigned to a new preferred CPU. Note that explicitly setting the preferred CPU overrides any default assignment of Fast Path ports to CPUs. This interface allows you the flexibility to load balance I/O activity over multiple CPUs in an SMP system. This is important because I/O activity can change over the course of a day or week.

> The $QIO passes in either a set containing one or more candidate CPUs, or zero as a wildcard value indicating the set of usable CPUs. If the candidate set contains only one CPU, you are explicitly designating the new preferred CPU. If the candidate set contains multiple CPUs, you are requesting use of the automatic preferred CPU assignment algorithm to select a suitable CPU from the candidate set.

> Including the IO$M_SYS_ASSIGNABLE modifier inhibits setting the selected CPU as the device's User Preferred CPU.

> The $QIO or the SET DEVICE/PREFERRED_CPU command will make a best effort to assign the port to a CPU. However, it is possible for this request to return failure for the following reasons:

> - There is no intersection between the candidate set and the node's set of usable CPUs.
> - There is resource contention. If after a reasonable effort the request is unable to acquire a key system resource, the request will fail. Some key resources include Fast Path spinlock, the CPU mutex, and a CPU transition lock.

> If the $QIO or SET DEVICE/PREFERRED_CPU returns failure, you should consider retrying either immediately or after a short delay. It is possible that a large number of ports were being reassigned, and the request failed due to resource contention.

> #### $IO_FASTPATH

> The $IO_FASTPATH system service performs operations on the set of Fast Path devices and CPUs enabled for Fast Path use. The $IO_FASTPATHW system service completes synchronously. That is, it returns after the operation is complete.

> The FP$K_BALANCE_PORTS function code specifies that the system service is to distribute the set of system assignable Fast Path ports across the intersection of a caller-supplied set of candidate CPUs.

### 10.2.3 Fast Path Restrictions

Fast Path restrictions include the following:

- Only high-volume I/Os are optimized.
  Fast Path streamlines the operation of high-volume I/O. I/O that does not meet the definition of high-volume is not optimized. A high-volume Fast Path I/O is characterized as follows:
  1. A virtual, logical, or physical read or write I/O without special I/O modifiers.
  2. An I/O request that is less than 64K bytes in size.
  3. An I/O issued when all I/O resources exist that are needed to perform the I/O.
- Send credits resource must be managed for DSA controllers.
  Applications seeking maximum performance must ensure the availability of sufficient I/O resources.
  The only I/O resource that a Fast Path user needs to be concerned about is send credits. Send credits are extended by DSA controllers to host systems and represent the maximum number of I/Os that can be outstanding at any given point in time. If an application sends an unlimited number of simultaneous I/Os to a controller, it is likely that some I/O will back up waiting for send credits. You can tell whether the send-credit limit is being exceeded by using the DCL command SHOW CLUSTER/CONTINUOUS, followed by an ADD CONNECTIONS, CR_WAIT command. Rapidly increasing credit-wait counts for the disk-class driver connections (a LOC_PROC_NAME name of VMS$DISK_CL_DRVR) is a sign that an application may be incurring send-credit waits.
  To ensure sufficient send credits, some controllers, like the HSC and HSJ, allow the number of send credits to vary. However, not all controllers have this flexibility, and different controllers have different send-credit limits. The best workaround is to know your application access patterns and look for send credit waits. If the number of send credits is being exhausted on one node, then add another controller to spread the load over multiple controllers. An alternative is to rework the application to load balance controller activity throughout the cluster, spreading a given controller's disk load over multiple nodes and allowing an application to exceed the send credits allotted to one node.

## Appendix A
I/O Function Codes

This appendix lists the function codes and function modifiers defined in the $IODEF macro. The arguments for these functions are also listed.

## A.1 ACP-QIO Interface Driver

This section lists the function codes and function modifiers for the ACP-QIO interface driver.

Functions | Arguments | Modifiers
--- | --- | ---
IO$_CREATE IO$_ACCESS IO$_DEACCESS IO$_MODIFY IO$_DELETE IO$_ACPCONTROL | P1---FIB descriptor address P2---file name string address P3---result string length address P4---result string descriptor address P5---attribute list address | IO$M_CREATE 1 IO$M_ACCESS 1 IO$M_DELETE 2 IO$M_DMOUNT 3
IO$_MOUNT | None | None

1Only for IO$_CREATE and IO$_ACCESS
2Only for IO$_CREATE and IO$_DELETE
3Only for IO$_ACPCONTROL

QIO Status Returns |  |
--- | --- | ---
SS$_ACCONFLICT | SS$_ACPVAFUL | SS$_BADATTRIB
SS$_BADCHKSUM | SS$_BADFILEHDR | SS$_BADFILENAME
SS$_BADFILEVER | SS$_BADIRECTORY | SS$_BADPARAM
SS$_BADQFILE | SS$_BLOCKCNTERR | SS$_CREATED
SS$_DEVICEFULL | SS$_DIRFULL | SS$_DIRNOTEMPTY
SS$_DUPDSKQUOTA | SS$_DUPFILENAME | SS$_ENDOFFILE
SS$_EXBYTLM | SS$_EXDISKQUOTA | SS$_FCPREADERR
SS$_FCPREWNDERR | SS$_FCPSPACERR | SS$_FCPWRITERR
SS$_FILELOCKED | SS$_FILENUMCHK | SS$_FILEPURGED
SS$_FILESEQCHK | SS$_FILESTRUCT | SS$_FILNOTEXP
SS$_HEADERFULL | SS$_IBCERROR 1 | SS$_IDXFILEFULL
SS$_ILLCNTRFUNC | SS$_NODISKQUOTA | SS$_NOMOREFILES
SS$_NOPRIV | SS$_NOQFILE | SS$_NOSUCHFILE
SS$_NOTAPEOP | SS$_NOTLABELMT | SS$_NOTPRINTED 1
SS$_NOTVOLSET | SS$_OVRDSKQUOTA | SS$_QFACTIVE
SS$_QFNOTACT | SS$_SERIOUSEXCP | SS$_SUPERSEDE
SS$_TAPEPOSLOST | SS$_TOOMANYVER | SS$_WRITLCK
SS$_WRONGACP |  |

1The second longword of the IOSB contains a job controller status code.

## A.2 Disk Drivers

This section lists the function codes and function modifiers for the disk drivers.

Functions | Arguments | Modifiers
--- | --- | ---
IO$_READVBLK IO$_READLBLK IO$_READPBLK 4 IO$_WRITEVBLK IO$_WRITELBLK IO$_WRITEPBLK 4 | P1---buffer address P2---byte count P3---disk address | IO$M_INHSEEK 1 IO$M_DATACHECK 2 IO$M_DELDATA 3 IO$M_INHRETRY IO$M_ERASE 5
IO$_WRITECHECK 2 | P1---buffer address P2---byte count P3---disk address | None
IO$_SENSECHAR IO$_SENSEMODE IO$_PACKACK IO$_AVAILABLE IO$_UNLOAD | None | None
IO$_SEARCH | P1---read/write head position | None
IO$_SEEK 4 | P1---seek to specified cylinder | None
IO$_FORMAT | P1---RX02 density | None
IO$_SETPRFPATH | P1---node or HSx name | IO$_FORCEPATH
IO$_CREATE IO$_ACCESS IO$_DEACCESS IO$_MODIFY IO$_DELETE IO$_ACPCONTROL | P1---FIB descriptor address P2---file name string address P3---result string length address P4---result string descriptor address P5---attribute list address | IO$M_CREATE 6 IO$M_ACCESS 6 IO$M_DELETE 7 IO$M_DMOUNT 8

1Only for IO$READPBLK and IO$_WRITEPBLK (not for TU58, RX01, RX02, RB02, or RL02)
2Not for RX01 and RX02
3Only for IO$_WRITEPBLK on RX02
4Not for DSA disks
5Only for write functions
6Only for IO$_CREATE and IO$_ACCESS
7Only for IO$_CREATE and IO$_DELETE
8Only for IO$_ACPCONTROL

QIO Status Returns
SS$_ABORT | SS$_CANCEL | SS$_CTRLERR
SS$_DATACHECK | SS$_DATAOVERUN | SS$_DRVERR
SS$_FORCEDERR | SS$_FORMAT | SS$_ILLIOFUNC
SS$_IVADDR | SS$_IVBUFLEN | SS$_MEDOFL
SS$_NONEXDRV | SS$_NORMAL | SS$_OPINCOMPL
SS$_PARITY | SS$_RCT | SS$_RDDELDATA
SS$_TIMEOUT | SS$_UNSAFE | SS$_VOLINV
SS$_WASECC | SS$_WRITLCK |

## A.3 Magnetic Tape Drivers

This section lists the function codes and function modifiers for the magnetic tape drivers.

Functions | Arguments | Modifiers
--- | --- | ---
IO$_READVBLK IO$_READLBLK IO$_READPBLK | P1---buffer address P2---byte count | IO$M_DATACHECK 1 IO$M_INHRETRY IO$M_REVERSE 3
IO$_WRITEVBLK IO$_WRITELBLK IO$_WRITEPBLK | P1---buffer address P2---byte count | IO$M_DATACHECK 1 IO$M_INHRETRY IO$M_INHEXTGAP 2 IO$M_NOWAIT 8 IO$M_ERASE 7
IO$_SETMODE IO$_SETCHAR | P1---characteristics buffer address P2---characteristics buffer length 9 |
IO$_CREATE IO$_ACCESS IO$_DEACCESS IO$_MODIFY IO$_ACPCONTROL | P1---FIB descriptor address P2---file name string address P3---result string length address P4---result string descriptor address P5---attribute list address | IO$M_CREATE 4 IO$M_ACCESS 4 IO$M_DMOUNT 5
IO$_SKIPFILE | P1---skip n tape marks | IO$M_ALLOWFAST 10 IO$M_INHRETRY IO$M_NOWAIT 8
IO$_SKIPRECORD | P1---skip n blocks | IO$M_INHRETRY IO$M_NOWAIT 8
IO$_REWIND IO$_REWINDOFF IO$_UNLOAD | None | IO$M_INHRETRY IO$M_NOWAIT IO$M_RETENSION
IO$_WRITEOF | None | IO$M_INHEXTGAP 2 IO$M_INHRETRY IO$M_NOWAIT 8
IO$_SENSEMODE IO$_SENSECHAR | P1---characteristics buffer address 9 P2---characteristics buffer length 9 | IO$M_INHRETRY
IO$_DSE 6 IO$_PACKACK IO$_AVAILABLE | None | None

1Not for TS04 and TU80
2Only for TE16, TU45, and TU77
3Not for TK50
4Only for IO$_CREATE and IO$_ACCESS
5Only for IO$_ACPCONTROL
6Only for TU78, TU81, TA81, and TA78
7IO$M_ERASE takes no arguments; only for IO$_WRITELBLK and IO$_WRITEPBLK on TMSCP drives.
8Only for TU81-Plus drives
9Only for TMSCP drives
10Only for local SCSI drives

QIO Status Returns |  |
--- | --- | ---
SS$_ABORT | SS$_CANCEL | SS$_CTRLERR
SS$_DATACHECK | SS$_DATAOVERUN | SS$_DEVOFFLINE
SS$_DRVERR | SS$_ENDOFFILE | SS$_ENDOFTAPE
SS$_ENDOFVOLUME | SS$_FORMAT | SS$_ILLIOFUNC
SS$_MEDOFL | SS$_NONEXDRV | SS$_NORMAL
SS$_OPINCOMPL | SS$_PARITY | SS$_SERIOUSEXCP
SS$_TIMEOUT | SS$_UNSAFE | SS$_VOLINV
SS$_WRITLCK |  |

Previous | Next | Contents | Index

| privacy and legal statement |
6136PRO_036.HTML