# hp Reliable Transaction Router
Application Design Guide

Previous | Contents | Index

## Administering Transaction Timeouts

RTR provides a relatively simple way to administer a transaction timeout in the server. Use of timeout values on the `Receive()` method lets a server application specify how long it is prepared to wait for the next message. (Of course, the server should be prepared to wait forever to get a new transaction or for the result of an already-voted transaction.)

One way to achieve this would be to have a transaction controller-specific global variable, say, called `SERVER_INACTIVITY_TIMEOUT` , which is set to the desired value (in milliseconds-that is, use a value of 5000 to set a 5-second timeout). Note that this timeout value should be used after receiving the first message of the transaction. The value should be reset to `RTR_NO_TIMOUTMS` after receiving the `rtr_mt_prepare` message. Whenever the Receive method completes with an `RTR_STS_TIMOUT` , the server transaction controller calls RejectTransaction to abort the partially processed transaction. This would prevent transactions from occupying the server process beyond a reasonable time.

## Two-Phase Commit

A prepared application votes its intention using the AcceptTransaction method. An application that does not agree to commit to this transaction votes with the RejectTransaction method. This is the first (or prepare) phase of the two-phase commit process.

### Initiating the Prepare Phase

The two-phase commit process with the C++ API is initiated by the client application when it issues a call to RTR indicating that the client "accepts" the transaction. This does not mean that the transaction is fully accepted, only that the client is prepared to accept it. RTR then asks the server applications participating in the transaction if they are prepared to accept the transaction. A server application that is prepared to accept the transaction votes its intention by issuing the AcceptTransaction method, an "accept" vote. A server application that is not prepared to accept the transaction issues the RejectTransaction method, a "not accept" vote. Issuing all votes concludes the prepare phase.

RTR provides an optional message to the server, OnPrepareTransaction, to expose the prepare phase. This message indicates to the server that it is time to prepare any updates for a later commit or rollback operation.

### Commit Phase

In the second phase of the commit, RTR collects votes from all the servers. If they are all votes to accept, then RTR tells all participant servers that they can now commit the transaction to the database. RTR also informs the client that the transaction has completed successfully. If any server rejects the transaction, all participants are informed of this and the database is left unchanged. Your application can use the level of participation that makes the most sense for your business and operations needs.

### Explicit Accept, Explicit Prepare

To request an explicit accept and explicit prepare of transactions, the server receives both prepare and accept messages. The server then explicitly accepts or rejects a transaction when it receives the prepare message. The transaction sequence for an explicit prepare and explicit accept is as follows:

Client | RTR | Server
--- | --- | ---
SendApplicationMessage | -> rtr_mt_msg1 | -> Receive
AcceptTransaction | -> rtr_mt_prepare | -> Receive
|  | <- AcceptTransaction
Receive | <- rtr_mt_accepted | -> Receive

With explicit transaction handling, the following steps occur:

- The server application waits for a message from the client application.
- The server application receives the `rtr_mt_prepare` request message from RTR.
- The server application issues the accept or reject.

A participant can reject the transaction up to the time RTR has sent the `rtr_mt_prepare` message to the server using the AcceptTransaction method and the AcceptTransaction method is executed. Once the client application has used the AcceptTransaction method, the result cannot be changed.

### Implicit Prepare, Explicit Accept

The sequence for an implicit prepare and explicit accept is as follows:

Client | RTR | Server
--- | --- | ---
SendApplicationMessage | -> rtr_mt_msg1 | -> Receive
AcceptTransaction | -> | <- AcceptTransaction
Receive | <-rtr_mt_accepted | -> Receive

In certain database applications, where the database manager does not let an application explicitly prepare the database, transactions can simply be accepted or rejected. For server optimization, the server can signal its acceptance of a transaction with either the SetAcceptTransaction method, or with the client calling the SetAcceptTransaction method. This helps minimize network traffic for transactions by increasing the likelihood that the data message and the RTR accept message will be sent in the same network packet.

### Server Transaction States

The server transaction states depend on whether the transaction is in prepare or commit phase. Table 4-4 lists server transaction states in the prepare phase.

Table 4-4 Prepare Phase Server States
State | Meaning
--- | ---
RECEIVING | This state represents the server not yet voting on the transaction.
VOTING | The transaction state changes to VOTING when the client has accepted the transaction and RTR has asked the server to vote but the server has not yet answered.

Table 4-5 lists server transaction states in the commit phase.

Table 4-5 Commit Phase Server States
State | Meaning
--- | ---
COMMIT | The server commits the transaction after receiving a message from RTR. If all servers vote to accept (AcceptTransaction), all participants receive a commit message.
ABORT | The server aborts the transaction after receiving a message from RTR. If any server votes to abort (RejectTransaction), all participants receive an abort message.

### Router Transaction States

The transaction states for the router depend on whether the transaction is in prepare or commit phase. Table 4-6 lists states in the prepare phase.

Table 4-6 Prepare Phase Router States
State | Meaning
--- | ---
SENDING | The router state changes to VREQ except on a failed transaction, in which case it changes to ABORTING.
VREQ | This state represents RTR waiting for the server to vote by issuing AcceptTransaction or RejectTransaction. Once a vote is received, the state changes to either COMMITTING or ABORTING.

# Transaction Recovery

When a transaction fails in progress, RTR provides recovery support using RTR replay technology. RTR, as a distributed transaction manager, communicates with a database resource manager in directing the two-phase commit process. Table 4-7 lists the typical server application transaction sequences for committing a transaction to the database. The sequence depends on which processing model you implement, polling or event driven.

Table 4-7 Typical Server Application Transaction Sequences
Polling Model | Event-Driven Model
--- | ---
1. RTRServerTransactionController:: Receive(rtr_mt_msg1) | 1. RTRServerTransactionController:: Receive(rtr_mt_msg1)
2. SQL update | 2. RTRServerMessageHandler:: OnInitialize()
3. RTRServerTransactionController:: AcceptTransaction() | 3. RTRServerMessageHandler:: OnApplicationMessage()
4. RTRServerTransactionController:: Receive(rtr_mt_accepted) | 4. SQL update
5. SQL commit | 5. RTRServerTransactionController:: AcceptTransaction()
6. RTRServerTransactionController:: AcknowledgeTransactionOutcome() | 6. RTRServerTransactionController:: Receive(rtr_mt_accepted)
| 7. RTRServerMessageHandler:: OnAccepted()
| 8. SQL commit
| 9. RTRServerTransactionController:: AcknowledgeTransactionOutcome()

## Recovery Examples

The impact of a crash on application recovery depends on where in the process the crash occurs. RTR handles the recovery, with the assistance of the application.

The typical server application transaction sequence using the event-driven processing model is as follows:

1. Receive(OnInitialize,
   OnApplicationMessage)
2. SQL update
3. AcceptTransaction
4. Receive(OnAccepted)
5. SQL commit
6. AcknowledgeTransactionOutcome

RTR transaction recovery summarized:

- RTR always replays the transaction with `rtr_mt_msg1` if the crash occurs between steps 1 and 3.
- With transaction manager coordination, RTR does not replay the transaction after step 3. The transaction is completed transparently to the application.
- Without transaction manager coordination, RTR replays the transaction with OnUncertainTransaction if the crash occurs between steps 3 and 6.

### Recovery: Before Server Accepts

If the failure occurs before the server accepts a transaction, the sequence is as follows:

1. Receive(OnInitialize, OnApplicationMessage)
   failure
2. SQL update
3. AcceptTransaction
4. Receive(OnAccepted)
5. SQL commit
6. Receive

If a crash occurs before the server accepts a transaction (between steps 1 and 3):

- With no database replication: the transaction is aborted and the database is rolled back.
- With a concurrent server: the database is rolled back and the transaction is replayed to another instance of the server.
- With a standby server: the database is rolled back and the transaction is replayed to the standby.
- With a shadow server: the shadow server completes the transaction as a lone member. On recovery, the database is rolled back and the transaction is replayed.

If another server (concurrent or standby) is available, RTR replays the transaction to that other server.

### Recovery: After Server Accepts

If the failure occurs after a server accepts a transaction, the sequence is as follows:

1. Receive(OnInitialize, OnApplicationMessage)
2. SQL update
3. AcceptTransaction
   failure
4. Receive(OnAccepted)
   failure
5. SQL commit
6. Receive

If a failure occurs after the AcceptTransaction method is executed, but before the SQL Commit, the transaction is replayed. The type of the first message is then `rtr_mt_uncertain` when the server is restarted. Servers should check to see if the transaction has already been executed in a previous presentation. If not, it is safe to re-execute the transaction because the database operation never occurred.

After the failure where the server accepts a transaction, but before the database is committed (between steps 3 and 5) the following occurs:

- With no database replication: the database is rolled back and the transaction is replayed as uncertain. The server should re-execute the transaction.
- With a concurrent server: the database is rolled back and the transaction is replayed as uncertain. The server should re-execute the transaction.
- With a standby server: the database is rolled back and the transaction is replayed as uncertain. The server should re-execute the transaction.
- With a shadow server: the shadow server completes the transaction as a lone member. When the failed server returns, the database is rolled back and transaction is replayed as uncertain. The server should re-execute the transaction.

If a failure occurs after the transaction has been accepted, but before it has been committed, the message is `rtr_mt_uncertain` when the server is restarted. It is safe to re-execute the transaction since the database commit operation never occurred.

### Recovery: After Database is Committed

If the failure occurs after the database is committed (for example, after the SQL commit but before receipt of a message starting the next transaction), RTR does not know the difference. The sequence is as follows:

- Receive(OnInitialize, OnApplicationMessage)
- SQL update
- AcceptTransaction
- Receive(OnAccepted)
- SQL commit
- Receive

If failure occurs after an SQL commit is made, but before the receipt of a message starting the next transaction (between steps 5 and 6), the sequence is as follows:

- With no database replication: the transaction is replayed as uncertain. The server should ignore the transaction.
- With a concurrent server: the transaction is replayed as uncertain. The server should ignore the transaction.
- With a standby server: the transaction is replayed as uncertain. The server should ignore the transaction.
- With a shadow server: the shadow server completes the transaction as a lone member. On recovery, the transaction is replayed as uncertain. The server should ignore the transaction.

In this case, the transaction should not be re-executed because the database commit operation has already occurred.

### Recovery: After Beginning a New Transaction

If the failure occurs after executing a Receive method to begin a new transaction, RTR assumes a successful commit (if the Receive occurs after receiving the `rtr_mt_accepted` message) and will forget the transaction. There is no replay following these events. The sequence is as follows:

1. Receive(OnInitialize, OnApplicationMessage)
2. SQL update
3. AcceptTransaction
4. Receive(OnAccepted)
5. SQL commit
6. Receive
   failure

If a crash occurs after a Receive call is made to begin a new transaction (after step 6), the sequence is as follows:

- With no database replication: No replay. Prior transaction was forgotten.
- With a concurrent server: No replay. Prior transaction was forgotten.
- With a standby server: No replay. Prior transaction was forgotten.
- With a shadow server: No replay. Prior transaction was forgotten.

As an application design suggestion, the application can maintain the list of transaction identifiers (TID) on a per-process, per-transaction controller basis to keep the list from growing infinitely.

## Exception Transaction Handling

RTR keeps track of how many times a transaction is presented to a server application before it is VOTED. The rule is: three strikes and you're out! After the third strike, RTR rejects the transaction with the reason `RTR_STS_SRVDIED` . The server application has committed the transaction and the client believes that the transaction is committed. The transaction is flagged as an exception and the database is not committed. Such an exception transaction can be manually committed if necessary. This process eliminates the possibility that a single rogue transaction can crash all available copies of a server application at both primary and secondary sites.

Application design can change this behavior. The application can specify the retry count to use when in recovery using the SetRecoveryRetryCount method in the RTRBackEndPartitionProperties class, or the system administrator can set the retry count from the RTR CLI with the SET PARTITION command. If no recovery retry count is specified, RTR retries replay three times. For recovery, retries are infinite. For more information on the SET PARTITION command, refer to the Reliable Transaction Router System Manager's Manual; for more information on the SetRecoveryRetryCount method, refer to the Reliable Transaction Router C++ Foundation Classes manual.

When a node is down, the operator can select a different backend to use for the server restart. To complete any incomplete transactions, RTR searches the journals of all backend nodes of a facility for any transactions for the key range specified in the server's RegisterPartition call.

## Coordinating Transactions

RTR provides two mechanisms for coordinating RTR transactions with database transactions (or database managers): transaction management coordination (XA, DECdtm) and RTR replay technology.

### Integration of RTR with Resource Managers

When RTR is used with a resource manager (database or DB manager), two transactions are defined: an RTR transaction and that of the resource manager. These must be coordinated to ensure that a transaction committed by the resource manager is also committed by RTR. The same applies to rejected transactions.

### Distributed Transaction Model

A distributed transaction manager provides a link for RTR to communicate with the resource manager in directing the two-phase commit process. Without the transaction manager, there is no path for RTR to communicate with a resource manager, so RTR must rely on the application for this communication. In general, two-phase commit coordination with the resource manager is not available to application programs.

Without a transaction manager, RTR uses replay technology to handle coordination with a resource manager. With a transaction manager, the transaction manager is the common agent to link the two transactions.

## Broadcast Messaging Processes

To use broadcast messaging with the C++ API, client and server applications can overload their event handlers.

To enable communication between two applications of the same type, create a second transaction controller of the other type. Messaging destination names can include wildcards, enabling flexible definition of the subset of recipients for a particular broadcast.

Use the SendApplicationEvent method to broadcast a user-event message. Broadcast types include user events and RTR events; both are numbered and can be named.

### User Events

A user broadcast is named or unnamed. An unnamed broadcast performs a match on the user event number. A named broadcast performs a match on both user event number and recipient name, a user-defined string. Named broadcasts provide greater control over who receives a particular broadcast.

Named events specify an event number and a textual recipient name. The name can include wildcards (% and *).

For all unnamed events, specify the evtnum field and `RTR_NO_RCPSPC` as the recipient name.

For example, the following code specifies named events for all subscribers:

`SendApplicationEvent( evUserEventNumber, "*", RTR_NO_MSGFMT)`

For a broadcast to be received by an application, the recipient name specified by the subscribing application on its RegisterFacility method for clients and RegisterPartition method for servers must match the recipient specifier used by the broadcast sender on the SendApplicationEvent call.

Note `RTR_NO_RCPSPC` is not the same as "*".

An application receives broadcasts with the Receive method. A message type returned in the RTRData buffers informs the application of the type of broadcast received. For example,

`Receive(...pmsg,maxlen,...pmsgsb);`

The user event would be in `msgsb.msgtype == rtr_mt_user_event` . User broadcasts can also contain a broadcast message. This message is returned in the message buffer provided by the application. The size of the user's buffer is determined by the maxlen field. The number of bytes actually received is returned by RTR in the msglen field of the message status block. Table 4-8 summarizes these rules.

Table 4-8 C++ API Named and Unnamed Broadcast Events
Broadcast Type | Match On | Specify
--- | --- | ---
Named | user event number | evtnum field
| recipient name | `rcpspc=` subscriber (for all subscribers, use `="*"` )
Unnamed | user event number | evtnum field
|  | `RTR_NO_RCPSPC`

### RTR Events

RTR delivers status information to which client and server applications can subscribe. Status information is delivered as messages, where the type of each message is an RTR event.

RTR events are numbered. The base value for RTR events is defined by the symbol `RTR_EVTNUM_RTRBASE` ; its maximum value is defined by the symbol `RTR_EVTNUM_RTRMAX` . RTR events and event numbers are listed in the Programming with the C++ API chapter of this guide and in the RTR header file `rtrapi.h` .

An application can subscribe to RTR events to receive notification of external events that are of interest to the application. For example, a shadow server may need to know if it is a primary or a secondary server to perform certain work, such as uploading information to a central database, that is done at only one site.

To subscribe to all RTR events, use the range indicators `RTR_EVTNUM_RTRBASE` and `RTR_EVTNUM_RTRMAX` .

RTR events are delivered as messages of type `rtr_mt.rtr.event` . Read the message type to determine what RTR has delivered. For example,

`rtr_status_t Receive( *pRTRData )`

The event number, evtnum, is returned in the RTRData. The following RTR events return key range data back to the client application:

- `RTR_EVTNUM_KEYRANGEGAIN`
- `RTR_EVTNUM_KEYRANGELOSS`

In application design, you may wish to consider creating separate facilities for sending broadcasts. By separating broadcast notification from transactional traffic, performance improvements can be substantial. Facilities can further be reconfigured to place the RTR routers strategically to minimize wide-area traffic.

A server application can expect to see a primary or secondary event delivered only in certain transaction states. For more detail, see the state diagrams in Appendix C, Server States.

Previous | Next | Contents | Index