DB Change Data Reader

The DB Change Data Reader stage allows you to select the columns to be included in the current jobflow, where the columns have the Change Data Capture feature enabled on them.

In the stage, you can create a Change Data Capture (CDC) Resource, which is the required data source table. The columns of the CDC resource on which the Change Data Capture feature is enabled are reflected using checkboxes.

Change Data Capture

The Change Data Capture feature enables capture of all changes made in a column. For each selected column, all inserts, updates, and deletions are captured.

Supported Databases

Currently, Spectrum Technology Platform supports the Change Data Capture (CDC) feature for MS SQL and Oracle databases only.

MS SQL

For MS SQL data sources, the Change Data Capture feature can be enabled or disabled for columns of tables from the backend. Refer to here for the necessary steps.

Note: The Change Data Capture feature is not supported in the Express edition of SQL Server.

Oracle

For Oracle data sources, Spectrum Technology Platform tracks the data changes in the table columns using the LogMiner utility of Oracle. CDC cannot be enabled or disabled on columns of tables in an Oracle data source through Spectrum Technology Platform.

The data changes resulting from insert, update, and delete queries are tracked and captured from an entered start date and time up to the current date and time. This entered start time is applicable during the first query execution with CDC switched on.

In subsequent executions on the same Oracle connection, data changes are captured from the time of the last execution till the current time.

Note: This is an incremental process. In the first capture, the changes are captured from the entered start date and time to the current date and time. In subsequent captures, the changes are captured from after the end date and time of the previous capture to the current date and time.