# hp Reliable Transaction Router
C Application Programmer's
Reference Manual

Previous | Contents | Index

## 2.12.3 Event Management by RTR

RTR manages both event routing and event delivery.

Event Routing

When an event subscription is created with the `rtr_open_channel` call, the event details are stored in a subscriber database on all routers. When an event is triggered, notification is delivered to all routers connected to that system in that facility. The routers then check their subscriber database for any systems that have subscribed to that event. If one or more subscribers are located, and the subscribers are currently attached to this router, then the router broadcasts the message to the subscribers. If no subscriber is located, then the message is discarded.

Event Delivery

RTR reliably delivers RTR transactions and RTR events. The delivery of User Events on a properly configured system is reliable, but RTR Flow Control manages delivery of User Events if the subscriber cannot process events as quickly as they are delivered. Flow Control is RTR's message traffic governor that helps affected systems to manage spikes in message traffic. For more detail on RTR Flow Control, refer to the Reliable Transaction Router System Manager's Manual.

When a User Event is triggered, a broadcast that includes message data is routed to the subscriber system. User Events, along with RTR Events and transactions, are placed into an incoming message queue on the destination system until the subscriber application executes an `rtr_receive_message` call to receive the message into the application. If too many messages are sent to the destination system, then the RTR Flow Control feature will be activated.

Flow Control may then force the sending application to wait awhile in the next RTR call that sends data, or it may discard broadcasts from the message queue, until the message queue length reduces and Flow Control allows new broadcasts to be sent to the destination system. Because User Event broadcasts are usually used for streaming information such as the periodic update of a price table, RTR does not store event messages that are impacted by Flow Control for later processing. This technique would cause the application to spend time viewing stale data. Instead, RTR Flow Control may discard the message to help relieve the messaging backlog, and will rely on a future message delivery to supply the updated information.

Design issue: Because of the possibility that a User Event message could become delayed or discarded due to Flow Control, User Events should not be used for delivering information that is of a business critical nature, including information that previously was, or later will be, used in a transaction. To compensate for the possibility of a discarded message, the application developer may consider adding a sequence number to the event message and providing a read-only transaction in the application to detect and request retransmission of any discarded broadcast data from the sender.

Overhead of Using Events

Delivery of User Events is based upon the registration databases that are kept on the routers. The event is delivered from the sender to all connected routers, which means each event triggers a message traffic load of 1 (for a FE sender) or the number of routers (for a BE sender). The event is then propagated by the routers to all subscribers, creating message traffic of 0 or the number of systems with subscribers to the event.

Design Issue: Processing event messages does consume some system resources and could impact overall performance. If system resources become constrained, RTR Flow Control may become active, thus reducing the RTR throughput on the affected systems. Care should be exercised to provide enough system resources to handle the message load.

## 2.12.4 Event Troubleshooting

Several RTR MONITOR screens can be helpful in troubleshooting events, as described below. Sample screens are available in the Reliable Transaction Router System Manager's Manual.

Monitoring Events

User Event traffic (broadcasts) may be monitored specifically for each node using the MONITOR BROADCAST screen in RTR. This screen shows the total event throughput, along with a count of any discarded broadcasts.

The MONITOR FACILITY screen in RTR provides a combined summary of all RTR Events and User Events processed for each facility.

The SHOW CLIENT/FULL and SHOW SERVER/FULL commands in RTR are helpful for viewing the current event subscription list for a particular client or server, along with any channel name specified in the rcpnam parameter on the `rtr_open_channel` call.

Execution of `rtr_broadcast_event` calls and event message traffic in RTR can be monitored using the MONITOR CALLS screen in RTR. This screen shows the frequency of use of the `rtr_broadcast_event` call, and the number of RTR Events and User Events processed. If an event is in pending ("pend") status, it indicates that the event is waiting for an `rtr_receive_message` call to be performed.

The MONITOR ROUTING screen shows the transaction and broadcast throughput on the system. This display shows the number of events and also the rate over time during the monitoring interval.

The MONITOR STALLS screen is helpful to determine if RTR Flow Control is affecting a particular system. Flow Control stalls that have occurred are categorized by duration. Any stall that lasts more than 60 seconds results in a Link Drop entry. A Stall ("stll") entry in the far-right column indicates that a Flow Control stall is currently in progress on the link indicated. For the purposes of User Event broadcast delivery, any stall could indicate that a broadcast message could have been discarded.

It is possible to monitor additional details of RTR Flow Control by using the MONITOR CONGEST, MONITOR FLOW, and MONITOR TRAFFIC monitor screens in RTR.

# 2.13 Use of XA Support

Users need to register a resource manager first, to invoke RTR XA support when creating a facility. Please see the Reliable Transaction Router System Manager's Manual for more details about how to register and unregister resource managers.

In the server application, specify the flag `RTR_F_OPE_XA_MANAGED` and the underlying resource manager information when issuing the `rtr_open_channel` call. Once this flag is specified for a given RTR partition, all transactions running in that RTR partition are committed using the XA interface between RTR and the resource manager. When the partition is deleted or the resource manager is unregistered, RTR commits transactions running in this partition in a conventional manner.

Note When running RTR Version 4.0 with Oracle8, Version 8.1.5 is required.

# 2.14 RTR Applications in a Multiplatform Environment

Applications using RTR in a multiplatform (that is, mixed endian) environment with nonstring application data have to tell RTR how to marshall the data for the destination architecture. The sender of a message must supply both a description of the application data being sent and the application data itself. This description is supplied as the msgfmt argument to `rtr_send_to_server` , `rtr_reply_to_client` , and `rtr_broadcast_event` .

The default (that is, when no msgfmt is supplied) is to assume the application message is string data.

## 2.14.1 Defining a Message Format

The msgfmt string is a null-terminated ASCII string consisting of a number of field-format specifiers:

[field-format-specifier...]

The field-format specifier is defined as:

%[dimension]field-type

where:

Field | Description | Meaning
--- | --- | ---
% | indicates a new field description is starting |
dimension | is an optional integer denoting array cardinality (default 1) |
field-type | is one of the following: |
| Code | Meaning
| UB | 8 bit unsigned byte
| SB | 8 bit signed byte
| UW | 16 bit unsigned
| SW | 16 bit signed
| UL | 32 bit unsigned
| SL | 32 bit signed
| C | 8 bit signed char
| UC | 8 bit unsigned char
| B | boolean

For example, consider the following data structure:

         typedef struct {
         rtr_uns_32_t first ;
         rtr_sgn_32_t second ;
         char  str[12] ;
     } example_t ;

The msgfmt for this structure could be " `%ul%sl%12c` ".

The transparent data type conversion of RTR does not support certain conversions (for example, floating point). Convert these to another format such as character string.

### 2.14.1.1 Data Types

Data types supported by RTR are:

- Unsigned
- Signed
- Char
- Boolean

### 2.14.1.2 Alignment

Alignment of data on byte boundaries depends on several factors, including the compiler used in creating an application. RTR's data marshalling software manages these alignments.

# 2.15 Application Design and Tuning Issues

This section addresses some considerations for design and tuning, including:

- Transactions that can cause server failure
- Transaction grouping and database applications
- Transaction sequence and shadow servers
- Transaction independence

## 2.15.1 Transactions That Can Cause Server Failure

It is possible for a "rogue" client transaction, due to a user application bug, to "kill" the server process. If RTR were to reapply this transaction indefinitely, all available servers would be destroyed. To avoid a transaction killing all server processes, the following mechanism is implemented:

- A transaction for which no `rtr_accept_tx` has been called by a server is aborted after it has caused the death of three concurrent servers to which it has been presented. The transaction abort status reported to the client is RTR_STS_SRVDIED. Retry count for transactions that have not been voted on is three; for transactions that have been voted on, retry count can be limited with the RTR command SET PARTITION/RECOVERY_RETRY_COUNT (default: unlimited).
- An RTR error log message with the same status is also written on the backend where the server deaths occurred.

The limitation of this feature to transactions that have not yet been accepted prevents possible transaction inconsistencies that could otherwise arise between client and server(s), and on shadow secondary sites. Thus a server application should complete any necessary validation of client transaction messages before accepting the transaction, to take advantage of this feature.

## 2.15.2 Transaction Grouping and Database Applications

RTR generates commit sequence numbers (CSN) for each transaction committed on the primary site. Concurrent servers can have several transactions assigned to a single CSN value. Transactions with the same CSN are understood by RTR to be independent, and hence their relative commit ordering to the database does not violate the serializability requirements of transactions.

For purposes of throughput, RTR attempts to group as many transactions as possible into a single CSN during a given vote cycle. (Grouped transactions are only those that explicitly vote (that is, call `rtr_accept_tx` on the server.)

The vote cycle completes as soon as RTR is ready to ask a server to commit the next transaction. For this mechanism to work correctly with the application, RTR places the following restriction on the server design:

> A server must obtain an exclusive lock on any resource that another concurrent server may be accessing for a different transaction before it issues the call to `rtr_accept_tx` .

Database applications, in general, comply with this requirement. If the database management software allows "dirty reads," the application should apply this rule explicitly, so that RTR can correctly serialize transactions during shadowing or other recovery. Failure to comply with this rule can cause unsynchronised copies of shadow databases.

## 2.15.3 Transaction Sequence and Shadow Servers

When using a facility having a shadow site and two or more partitions, the transaction sequence is the same at both shadow sites within a single partition only. Sequences across partitions are not preserved. For example, suppose the following transactions are executed on half of a shadow site in the following chronological order:

> tx1_for_partition1
> tx2_for_partition1
> tx3_for_partition1
> tx1_for_partition2
> tx4_for_partition1

When replayed on the secondary, the order could be:

> tx1_for_partition1
> tx2_for_partition1
> tx3_for_partition1
> tx4_for_partition1
> tx1_for_partition2

Do not write your application to expect preservation of transaction serialization across partitions.

## 2.15.4 Transaction Independence

RTR normally assumes that each transaction processed by a given server depends on the transactions that particular server has previously accepted.

To keep the shadowed database identical to that on the primary, RTR controls the order in which the secondary executes transactions. The secondary is constrained to execute transactions in the same order as the primary. Under some circumstances, this can lead to the secondary sitting idle, waiting to be given a transaction to execute.

RTR provides a performance enhancement that may help some applications decrease idle time on the secondary, reducing the corresponding backlog. If the application knows that particular transactions are independent of the transactions previously received, then the application can set one of two flags listed in Table 2-9.

Table 2-9 Independent Transaction Flags
Flag | Meaning
--- | ---
RTR_F_ACC_INDEPENDENT | Set on an `rtr_accept_tx` call to indicate this transaction is independent.
RTR_F_REP_INDEPENDENT | Set on an `rtr_reply_to_client` call along with `RTR_F_REP_ACCEPT` to indicate this transaction is independent.

A transaction accepted with one of these flags can be started on the secondary while other transactions are still running. All transactions flagged with one of these flags must truly be independent of the transactions that have previously executed. They will execute in an arbitrary sequence on the secondary site.

If the server channel has been opened with `RTR_F_OPE_EXPLICIT` (explicit accept), then the `RTR_F_REP_INDEPENDENT` flag can only be used together with `RTR_F_REP_ACCEPT` . If the server channel has been opened with implicit accept, then using `RTR_F_REP_INDEPENDENT` implies using `RTR_F_REP_ACCEPT` .

An application can be written to create CSN boundaries to ensure independence. A transaction always receives a CSN, and the `INDEPENDENT` flag could be used to prevent the CSN from being incremented, so an application could be coded to force dependence between sets of transactions. This could be important in certain cases where transactions coming in at a particular time of day are independent of each other, but other transactions executed, say, at the end of the day, need to ensure that the day's transactions have been processed, and the following day's transactions need to ensure that the previous end-of-day processing has completed. For more details on user of independent transactions, refer to the discussion of CSNs in the Reliable Transaction Router Application Design Guide.

Previous | Next | Contents | Index