Welcome
Explore Data Integration. Connect different data sources and various stages used in Data Integration process.
Introduction
Learn about data management architecture and the star schema data warehouse design.
Connecting to Data Sources and Data Warehouses
Connect to different types of data sources.
Populating the Data Warehouse
Prepare your data for warehouse. Populate Time Dimension table, Dimension table, Fact table. Add time stamp to records in a Data Warehouse.
Updating the Data Warehouse
Define a Data Warehouse update schedule. Update a fact table. Use Global and Local cache for queries.
Accessing Stages Through Enterprise Designer
Access different stages through Enterprise Designer.
- Call Stored Procedure
- DB Change Data Reader
  The DB Change Data Reader stage allows you to select the columns to be included in the current jobflow, where the columns have the Change Data Capture feature enabled on them.
- DB Loader
  The DB Loader stage allows you to access and load data from/to databases configured in Spectrum Data Integration. This stage provides an interface to a high-speed data loading utility. Currently, the Spectrum Data Integration platform supports Oracle Loader, DB2 Loader, PostgreSQL Loader, and Teradata Loader.
- Field Parser
- Field Combiner
- Field Selector
- Generate Time Dimension
- Query Cache
- Query DB
- Query NoSQL DB
- Read From DB
- Read From File
- Read from Hadoop Sequence File
- Read From Hive File
- Read from HL7 File
- Read from NoSQL DB
- Read from SAP
- Read from Spreadsheet
- Read from Variable Format File
- Read From XML
- SQL Command
- Transposer
- Unique ID Generator
- Write to Cache
- Write to DB
- Write to File
- Write to Hadoop Sequence File
- Write to Hive File
- Write to NoSQL DB
- Write to Spreadsheet
- Write to Variable Format File
- Write to XML
- Date and Number Patterns
Configurations
Configure Oracle LogMiner, and HDFS connection for HA Cluster.
Optimizing Performance
Improve performance by determining an optimum fetch size. Learn about best practices for connecting to HDFS 3.x and Hive 2.1.1.

Defining a Unique ID

By default, the Unique ID Generator stage creates a sequential ID, with the first record having an ID of 0, the second record having an ID of 1, the third record having an ID of 2, and so forth. If you want to change how the unique ID is generated, follow this procedure.

In the Unique ID Generator stage, on the Rules tab, click Modify.
Choose the method you want to use to generate the unique ID.
For more information, see Unique ID Definition Methods.
Click OK.