Dataflow Designer's Guide

Content
Search Results
Loading, please wait ...

Loading

  • Welcome
  • Getting Started
    • Installing the Client Tools
    • Starting Enterprise Designer
    • A First Look at Enterprise Designer
    • My First Dataflow (Job)
    • My First Dataflow (Service)
    • Dataflow Templates
      • Creating a Dataflow Using a Template
    • Importing and Exporting Dataflows
  • Designing a Flow
    • Types of Flows
    • Flow Input
      • Defining Job Input
        • Managing malformed input records
      • Defining Service Input
        • Defining Input Fields for a Service or Subflow
        • Defining A Web Service Data Type
    • Fields
      • Flat and Hierarchical Data
        • Converting flat data to a list
      • Data Types
        • *Automatic Data Type Conversion
          • Setting Data Type Conversion Options for a Flow
          • Date and time patterns
          • Number Patterns
        • Changing a field's data type
      • Changing a field's name
      • Reserved Field Names
    • Control Stages
      • Aggregator
      • Broadcaster
      • Conditional Router
        • Configuring a Conditional Router
          • Using the Expression Builder
          • Writing a Custom Expression
      • Group Statistics
        • Operations
        • Output Columns
        • Pivot Tables
          • Creating a Pivot Table
      • Math
        • Using the Calculator
          • Using Functions and Constants
          • Using Conditional Statements
          • Using the Expressions Console
          • Using the Fields Control
          • Using the Preview Control
      • Record Combiner
      • Record Joiner
      • Sorter
        • Sorting Records with Sorter
      • Splitter
      • Stream Combiner
      • Transformer stage transform types
        • Changing the Order of Transforms
        • Creating a Custom Transform
        • Using a Mask Transform
      • Unique ID Generator
        • Defining a Unique ID
        • Using Algorithms to Augment a Unique ID
        • Defining a Non-Unique ID
    • Output stages (Sinks)
      • Defining Service Output
        • Defining A Web Service Data Type
      • Running an External Program
      • Terminating a Job Based on a Condition
      • Discarding records - Write to Null
    • Embedded flows
      • Grouping stages into an embedded flow
      • Editing an embedded flow
      • Using iteration with an embedded flow
      • Ungrouping an embedded flow
      • Converting an embedded flow to a subflow
    • Reports
      • Adding a standard report to a job
      • Setting report options for a job
      • Viewing reports
      • Using custom reports
    • Performance Considerations
      • Design guidelines for optimal performance
      • Stage Runtime Performance Options
        • Database Pool Size and Runtime Instances
        • Distributed Processing
          • Designing a flow for distributed processing
        • Running a Stage on a Remote Server
          • Troubleshooting Remote Server Errors
      • Optimizing Stages
        • Optimizing Matching
        • Optimizing Candidate Finder
        • Optimizing Transforms
        • Optimizing Write to DB
        • Optimizing Address Validation
        • Optimizing Geocoding
    • Flow Versions
      • Saving a Flow Version
      • Viewing a Flow Version
      • Editing a Flow Version
      • Editing Version Properties
      • Exposing a Version
  • Inspecting and Testing
    • Checking a Flow for Errors
    • Inspecting a flow
    • Testing a service with Management Console
  • Running a Flow
    • Running a Job or Process Flow
      • Running a Flow in Enterprise Designer
      • Running A Job from the Command Line
        • Overriding Job File Locations
        • Overriding the File Format at the Command Line
        • Using a Job Property File
      • Running a Process Flow from the Command Line
        • Using a Process Flow Property File
      • Scheduling a Flow
      • Triggering a Flow with a Control File
      • Viewing Flow Status and History
        • Downloading Flow History
      • Setting the Malformed Records Default
      • Setting Report Defaults
    • Exposing a Service
      • Exposing a Service as a Web Service
      • Exposing a Service for API Access
    • Runtime Options
      • Adding Flow Runtime Options
      • Specifying Default Service Options
      • Deleting flow Runtime Options
    • Configuring Email Notification for a Flow
  • Combining Flows into a Process Flow
    • Introduction to Process Flows
    • Designing Process Flows
      • Creating a Process Flow
      • Using a Variable to Reference a File
      • Adding Conditional Logic to a Process Flow
      • Deleting a Process Flow
      • Activities
        • Job
          • Overriding Input and Output Files
        • Clear Cache
        • Execute SQL
        • Load to Hive
          • Creating a Hive Connection
        • Run Hadoop MapReduce Job
          • Supported job configurations
          • Variables
        • Run Hadoop Pig
          • Hadoop Pig Operations
        • Run Program
          • Specifying Input and Output Files
          • Using a Control File with an External Program
        • Spark Sorter
          • Defining Fields In a Delimited Input File
          • Sorting Records
          • Variables
        • Submit Spark Job
          • Supported job configurations
          • Variables
        • Success
  • Creating Reusable Flow Components
    • Introduction to Subflows
    • Using a Subflow as a Source
    • Using a Subflow in the Middle of a Flow
    • Using a Subflow as a Sink
    • Modifying a Subflow
    • Deleting a Subflow
    • Exposing and Unexposing a Subflow
    • Converting a Stage to a Subflow
  • Sample Flows
    • Introduction
    • Integration between SugarCRM OnPremises and Microsoft Dynamics 365 Online
    • Integration between Salesforce and Oracle Eloqua
  • About Spectrum™ Technology Platform
    • What Is Spectrum™ Technology Platform?
    • Enterprise Data Management Architecture
    • Spectrum™ Technology Platform Architecture
    • Modules and Components