The Performance Log

The performance log contains details about how long it takes for a job or service to run. It includes overall performance information for the job or service as well as performance information for each stage in the job or service dataflow. You can use this information to identify bottlenecks in your dataflow by looking at the execution time and processing time for each stage. A large difference between execution time and processing time means that the stage is spending time waiting for data from upstream stages. This may indicate that an upstream stage is a bottleneck in the dataflow. Note that for sinks, a large difference between execution time and processing time does not necessarily indicate a performance problem because sinks typically have to wait for the first records from the rest of the dataflow.

To enable performance monitoring for a job or service, use the performancemonitor enabled set command in the Administration Utility.

The performance log is located on your Spectrumâ„¢ Technology Platform server in the following location:

SpectrumLocation\server\app\repository\logs\performance.log

The performance log contains one row for each run of a monitored job or service. It is a rolling log that consists of a maximum of five files. Each file is limited to 10 MB in size. Once this limit is reached, the oldest performance data is deleted when new performance data is logged.

Each entry in the performance log contains the following information.
Note: For ease of reading, line breaks and indentation are shown below. In the actual log, the entry is on one line.
Date Time [performance]
{
     "username":"UserName",
     "dataflowId":"DataflowName",
     "runMode":"BatchOrRealTime",
     "remote":TrueOrFalse,
     "elapsedTime":Nanoseconds,
     "stageInfo":[
          {
               "stageName":"Name",
               "stageLabel":"Label",
               "options":{
                    OptionsList
               },
               "recordsRead":Count,
               "recordsWritten":Count,
               "executionTime":Nanoseconds,
               "processingTime":Nanoseconds
          }
     ]
}

Where:

username
The user who executed the job or service.
dataflowID
The name of the service or job as defined in Enterprise Designer.
runMode
Indicates whether the log entry is for a job or a service. One of the following:
Batch
The log entry is for a job.
RealTime
The log entry is for a service.
remote
Indicates whether the job or service was executed in the local server or on a remote server. For jobs that contain one or more stages that run on a remote server, the performance log on the server running the job will indicate that the job is not remote (a value of "false") while the performance log on the remote server that executes one or more of the job's stages will indicate that the job is remote (a value of "true"). One of the following:
true
The job or service was executed on a remote server.
false
The job or service was executed on the local server.
elapsedTime
The time in nanoseconds that it took to execute the job or service request.
stageInfo
Lists execution information for each stage in the dataflow. The following information is listed for each stage:
stageName
The permanent name of the stage.
stageLabel
The user-defined name of the stage. The stage label is shown on the canvas in Enterprise Designer.
options
If any options were specified at runtime, those options and their settings are listed here.
recordsRead
The total number of records that passed into the stage through all the stage's input ports.
recordsWritten
The total number of records that the stage wrote to all its output ports.
executiontime
The amount of time from when the stage processed its first record to when it processed its last record. This includes the time the stage was idle while waiting for data from other stages in the dataflow.
processingtime
The amount of time the stage spent actively processing records, not including the time it was idle while waiting for other stages in the dataflow.