Manage Replication Pipeline

The replication pipelines displayed on the Replication Pipelines page shows the current health/status of all pipelines that have been finalized/deployed (not sure what wording we want to use). It also shows whether each pipeline has any edits in progress, and whether or not those edits are staged. It will also show pipelines that have not been finalized yet (which do not yet have a health/status) and those that are marked for deletion.

Create Replication Pipeline

You can create new data connections while creating a pipeline. Each pipeline also uses two data connections, currently, one for source and target.

Disclaimer: Due to the evolving nature, certain parts of this video may or may not be current with your version of the product.
  1. Enter general information about your pipeline.
    • Name of the pipeline
    • Description of the pipeline
    • Type of pipeline: Synchronize or Copy or Replication
      • Synchronize (the default) - copies a snapshot of a set of source tables from the database source and distributes it to the target. Data changed during the copy process is captured. When the copy process finishes, a change-based replication begins, applying all changes made during the copy up through the time the copy finished. Thereafter, changes made to the source data are detected, captured, and replicated to the Kafka target, synchronizing the table column data. Synchronize combines Copy and Replication activities.
      • Copy - Detects changes made to data, captures the changes, and applies them to the Kafka target.
      • Replication -
  2. Select a data connection that was defined in the Data Connections tab.
    Note: You can add a Source or Target connection if those connections are not currently available within the data connection that has been previously defined.
  3. Once the Source and Target connections are selected, you can define additional options.

Edit Replication Pipeline

You can edit an existing pipeline.
Note: You cannot edit an existing pipeline if another user is currently editing it.
  1. Click the next to a pipeline to open the menu.
  2. Select Edit.
    The Edit Pipeline page displays.
  3. Step through the wizard and update options associated with the pipeline.
    Note: If the pipeline was created when the table mapping feature was OFF and now the table mapping feature is ON, the mapping is updated now. All to one mapping will be updated to One to One mapping. If there is no topic available matching to the source table name at the kafka target, then the mapping stage displays a RED x icon indicating you must create the topic.
  4. Click Finish.
  5. Return to Replication Pipeline page and the message next to the pipeline states This replication pipeline has been modified but the configuration changes have not been staged or made active.
  6. Click at the top to open the context menu and select Apply Configuration Changes.
    The Apply Configuration Changes page opens.
  7. Select Stage configuration changes and/or Make Configuration Changes Active and click Apply.

Delete Replication Pipeline

You can delete an existing pipeline.
  1. Click the next to a pipeline to open the context menu.
  2. Select Delete.

Manage Capture

Enable capture for a replication pipeline source table adds triggers which continuously capture data changes on the table. Disable capture for a table removes those triggers, excluding the table from the capture requests. When replication is active for enabled tables, a copy of the changed data is transferred (replicated) to your target.
  1. Click the next to a pipeline to open menu and select Manage Capture.
    The Manage Capture page opens with the name of the data connection, the connection type, and replication pipeline.
  2. Click the check box next to the table with a capture status of Disable that you want to enable and click Enable.
    Note: You cannot enable tables if the associated project is stopped, the replication pipeline is started, or the data connection type is Db2 for z/OS.
  3. Optional. Click the check box next to the table with a capture status of Enable that you want to disable and click Disable.
    Note: You cannot disable tables if the associated project is stopped, the replication pipeline is started, or the data connection type is Db2 for z/OS.
  4. Select the option Display alerts to monitor progress to show alerts.
  5. Click Save.