Monitor Replication Pipelines

Replication pipelines used in a replication project can copy, synchronize, and replicate your data from source to a target stream. This screen allows the user to monitor details of a selected replication pipeline and perform certain actions based on the component. When you click the selected replication pipeline, additional details such as health of the pipeline, the current phase of the pipeline, required actions, and the statuses of the processes are displayed. For example, if the overall health status is Stopped ( ) that indicates all processes and replication pipelines for the project are stopped. This includes kernel processes on all runtime engines and replication processes for all replication pipelines. Capture can still be enabled. Staged but not active projects will appear stopped. You can also perform actions on the components. For example, by clicking () next to replication pipeline, you can select an item from the context menu.

View Replication Pipeline Details

You can view replication pipeline details for all active replication projects in Data Integration to help identify component status and data processing rates for individual transactions in the replication pipeline. For example, if replication rates change, view the replication pipeline details to compare the number of transactions and bytes processed at the source and target. You can access the replication pipeline details for information on the latest statistics and current statuses. The information included in the replication pipeline details varies depending on the type of replication pipeline. For example, not all details are valid for copy replication pipeline.
Disclaimer: Due to the evolving nature, certain parts of this video may or may not be current with your version of the product.
  1. To open a replication pipeline, click from the top of the navigation to view all details about the replication pipeline.
  2. Click Refresh to see the latest statistics, details, and statuses.
  3. Click Close.

Start Replication Pipeline

You can manually start a replication pipeline. When you start a replication pipeline, the following occurs:
  • If the pipeline is started for the first time:
    • any log readers are started
    • tables are enabled for capture
    • for replication, data is replicated to the target
    • for sync, the initial copy/load is done and then captured data is replicated to the target
    • If this a copy, no log readers are started, tables do not get enabled for capture (because this is just for copy) the data is copied to the target. Once the copy is completed the copy pipeline stops.
  • If the pipeline is started after being stopped and at least one table is enabled for capture
    • replication pipeline starts replicating data to the target
    • sync continues with replication or to recopy the data. Default to continue replication and NOT copy.

To start replication pipeline

  1. To open a replication project, click from the top of the navigation menu and select Start . The Start Replication Pipeline page opens.
  2. For synchronize data flows, to copy the full set of data to the target again, select Copy data before starting replication in synchronize data flows.
    Note: This is required the first time you start a synchronize replication pipeline.
  3. Do one of the following:
    • Click Start to start the replication pipeline. One of the following occurs:
      • If there are no errors, the window closes. Statuses for source server, replication, and the overall replication pipeline status show as Started. When the replication pipeline are restarted, the values are refreshed with the latest statistics.
      • If there are problems starting the replication pipeline(s), an error message displays alerting you to the problems.
    • Click Cancel to close the window without starting the replication pipeline.

Stop Replication Pipeline

You can manually stop a replication pipeline. Stopping the replication pipeline stops replication from the source database to the target for the enabled tables. You must stop a replication pipeline before you enable or disable data capture. You may also need to stop replication pipeline when the target is unavailable.
  1. To open a replication project, click from the top of the navigation menu and select Stop.
  2. The Stop Replication Pipeline page opens showing the name and type of each selected replication pipeline. If any selected replication pipeline are already stopped, their names display but are ignored during the stop process
    Note: If the replication project is currently deploying or is set to Ignored, the Stop button is unavailable
  3. Do one of the following:
    • Click Stop to stop the replication pipeline.
    • Click Cancel to close the window without stopping the replication pipeline.

Clear Backlog

When replication processes are active, changed data in enabled tables is captured and added as data rows to the replication backlog in the associated metabase to await replication to the target. Clearing the backlog purges (deletes) any data rows in the backlog waiting to be replicated. Clearing the backlog is sometimes necessary when replication activities cannot proceed; for example, when there are problems with the backlog draining automatically or there are data rows in the backlog that cannot be processed.
  1. To open a replication project, click from the top of the navigation menu and select Clear Backlog.
    The Clear Backlog dialog opens.
  2. Select the Confirm clearing the backlog option to clear the backlog rows.
    Note: This option is displayed if the number of backlogs rows is more than 100,000.
  3. Do one of the following:
    • Click Clear Backlog to close the dialog. When the backlog finishes clearing, the value in the Backlog Rows column displays 0 (zero). If a problem is encountered and the backlog cannot be cleared, an error message displays.
    • Click Cancel to close the dialog without clearing the backlog.

Metabases

Use this option to display the list of metabases associated with a pipeline.

Create a Metabase for Oracle for JDBC data connection

You can create a metabase when you are a creating a new pipeline if it is the first pipeline in the project that uses this particular source data connection. Additionally, you can create a metabase from the project level action menu.
  1. To open a replication project, click from the top of the action menu and select Metabase.
    The Metabase page opens.
  2. If a metabase is not present, click to add a new metabase.
    The New Metabase page opens.
  3. Enter the required information:
    OptionDescription
    Data connection Oracle JDBC data connections.
    Note: If there is only one Oracle JDBC data connection it is selected by default.
    Database server

    The address of the database server the metabase will be created on. This value is populated when the data connection is selected.
    Oracle administration user ID

    Oracle system admin user ID for source server. The value is SYS.
    Password

    Password for the oracle admin credentials, used to create the metabase. Password is not stored.
    Replication user ID
    A user will be created. For Oracle, this user must be unique to the metabase or project.
    Note: The user cannot be reused across projects.
    Password

    Password for the user ID.
    Metabase name

    Name of the metabase. The value is the same as the Replication user ID.

    Tablespace for metabase

    Specify the name of the tablespace to create for the metabase.
    Data file

    Specifies the file on the database server that is used to store the tablespace for the metabase
    Size

    Specifies the size of the file that contains the new tablespace you are creating, in Megabytes. The default value is 200.
    Temporary tablespace for metabase

    Specify the name of the tablespace in which to create the temporary metabase space.
    Temporary data file

    Specifies the file that is to store the temporary tablespace for the metabase.
    Size

    Specifies the size of the file that contains the new temporary tablespace you are creating, in Megabytes. The default value is 200.
  4. Click Create.

Create a Metabase for Db2 for IBM i source data connection

  1. To open a replication project, click from the top of the action menu and select Metabase.
    The Metabase page opens.
  2. If a metabase is not present, click to add a new metabase.
    The New Metabase page opens.
  3. Enter the required information:
    OptionDescription
    Data connection

    Db2 for IBM i JDBC data connections.

    Note: If there is only one Db2 for IBM I, JDBC data connection it is selected by default.
    Database server

    The address of the database server the metabase will be created on. This value is populated when the data connection is selected.
    Replication user ID

    A user chosen during the Change Selector (Log Reader) software installation on IBM i. For example, OMNIENT.
    Password

    Password for the user ID.
    Metabase library

    Installing the metabase will create the metabase library with this name. The metabase library must be unique for this project/source combination and must not used by any other replication project.
  4. Click Create.

Create a Metabase for Db2 for Linux, Unix, or Windows type data connection

  1. To open a replication project, click from the top of the action menu and select Metabase.
    The Metabase page opens.
  2. If a metabase is not present, click to add a new metabase.
    The New Metabase page opens.
  3. Enter the required information:
    OptionDescription
    Data connection

    Db2 for Linux, Unix, Windows JDBC data connections.

    Note: If there is only one Db2 for Linux, Unix, Windows, JDBC data connection it is selected by default.
    Database server

    The address of the database server the metabase will be created on. This value is populated when the data connection is selected.
    Db2 administrator user ID: Specifies the Db2 admin user ID.
    Password

    Password for the Linux, Unix, Windows server admin credentials, used to create the metabase. Password is not stored.
    Replication user ID

    A user ID to associate with the metabase.
    Password

    Password for the user ID.
    Metabase name

    Name of the metabase. The default value: RPUSER.
  4. Click Create.