Transactional Support

This section summarizes the transactional and high availability support for EventFlow operators and adapters.

EventFlow Entities

The table below summarizes the current transactional support status for all EventFlow entities.

The Transactional Support column has one of these values:

  • Full - complete support for transaction semantics when using transactional memory.
  • Deadlock Safe - entity correctly handles transactional deadlocks. Entity does not have any state that must be stored in transactional memory to achieve transaction semantics, so it is fully transaction safe.
  • Not Transaction Safe - entity is non-transactional and not deadlock safe. Entity state has no transaction isolation and is not rolled back on transactional rollback.

The High Availabilty Support column has one of these values:

  • Full - complete support for high-availabilty, including data sharing between nodes, when using transactional memory.
  • Backup - entity state can be backed up to another node in the cluster, but the data cannot be shared between nodes. If a node fails, when the node is restarted its entity state is automatically restored from a backup node.
  • Not Required - entity has no state that must be made highly available.
  • Possible Data Loss - entity has state that is only stored on the local node and cannot be replicated. This state is lost if a node fails.
Entity Transactional Support High Availability Support Comments
Aggregate Operator Full Backup
BSort Operator Full Backup
Custom Adapter Full Backup Use Operator.setManagedState>> and <<<Operator.getManagedState
Custom Aggregate Functions Full Backup Must directly configure Java classes for high-availability. See DataMapper configuration object.
Custom Operator Full Backup Use Operator.setManagedState and Operator.getManagedState
Decision Table Operator Deadlock Safe Not Required
Distributed Router Operator Deadlock Safe Not Required
Dynamic Variables Full Backup
Extension Point Operator Deadlock Safe Not Required
Filter Operator Deadlock Safe Not Required
Gather Operator Full Backup
Heartbeat Operator Full Backup
Iterator Operator Deadlock Safe Not Required
Join Operator Full Backup
Map Operator Deadlock Safe Not Required
Merge Operator Full Backup
Metronome Operator Full Backup
Parallel Region Queue Full Full See Parallel Region Queues.
Module Reference Operator Deadlock Safe Not Required
Pattern Operator Full Backup
Query Operator Deadlock Safe Not Required
Query Table Full Full
Sequence Operator Full Backup
Split Operator Deadlock Safe Not Required
Union Operator Deadlock Safe Not Required

Adapters

Input adapters are transaction safe; they are immune to transactional rollback. Data is made highly available by storing it in an entity, e.g. a query table, in EventFlow. No high availability is provided directly by input adapters.

Input adapters that use Operator.sendOutputAsync have a possible data loss exposure that cannot be closed since the queued work is not highly available.

Output adapters are not transaction safe unless they are communicating with a transactional server; they are not immune to transactional rollback. If a transaction is rolled back the data is sent twice to the external system. To avoid duplicate data output adapters should be put into a separate transactional memory parallel region to make them immune to transaction rollback.

Parallel Region Queues

In addition to optional transactional behavior, parallel region queues now also support:

  • Maximum queue depth.
  • High availability.

High availabilty is only supported when the queue is configured to be in transactional memory.

Maximum Queue Depth

The maximum number of tuples that can exist in a queue is now configurable.

Queues stored in heap memory will block the input thread when the maximum queue depth is hit. The input thread will be unblocked when the queue depth drains below the maximum value.

Queues stored in transactional memory will abort the current transaction when the maximum queue depth is exceeded.

The maximum queue depth is configured for an engine using this configuration property:

parallelRegionQueues.defaultMaxOutstandingTuples

The default configuration value is 0, which indicates no limit.

High Availability

Parallel region queues stored in transactional memory can be configured for high-availability.

Parallel region queues are associated with a data distribution policy using engine configuration. The data distribution policy must be defined in an application definition configuration (com.tibco.ep.dtm.configuration.application) associated with the application.

Currently only static data distribution policies are supported. The data mapper associated with the data distribution policy is always round-robin.

The data distribution policy is configured using this system property:

parallelRegionQueues.defaultDataDistributionPolicy