The Performance Log
The performance log contains details about how long it takes for a job or service to run. It includes overall performance information for the job or service as well as performance information for each stage in the job or service dataflow. You can use this information to identify bottlenecks in your dataflow by looking at the execution time and processing time for each stage. A large difference between execution time and processing time means that the stage is spending time waiting for data from upstream stages. This may indicate that an upstream stage is a bottleneck in the dataflow. Note that for sinks, a large difference between execution time and processing time does not necessarily indicate a performance problem because sinks typically have to wait for the first records from the rest of the dataflow.
To enable performance monitoring for a job or service, use the performancemonitor enabled set command in the Administration Utility.
The performance log is located on your Spectrumâ„¢ Technology Platform server in the following location:
SpectrumLocation\server\app\repository\logs\performance.log
The performance log contains one row for each run of a monitored job or service. It is a rolling log that consists of a maximum of five files. Each file is limited to 10 MB in size. Once this limit is reached, the oldest performance data is deleted when new performance data is logged.
Date Time [performance]
{
"username":"UserName",
"dataflowId":"DataflowName",
"runMode":"BatchOrRealTime",
"remote":TrueOrFalse,
"elapsedTime":Nanoseconds,
"stageInfo":[
{
"stageName":"Name",
"stageLabel":"Label",
"options":{
OptionsList
},
"recordsRead":Count,
"recordsWritten":Count,
"executionTime":Nanoseconds,
"processingTime":Nanoseconds
}
]
}
Where:
- username
- The user who executed the job or service.
- dataflowID
- The name of the service or job as defined in Enterprise Designer.
- runMode
- Indicates whether the log entry is for a job or a service. One of the following:
- Batch
- The log entry is for a job.
- RealTime
- The log entry is for a service.
- remote
- Indicates whether the job or service was executed in the local server or on a remote
server. For jobs that contain one or more stages that run on a remote server, the
performance log on the server running the job will indicate that the job is not remote
(a value of "false") while the performance log on the remote server that executes one or
more of the job's stages will indicate that the job is remote (a value of "true"). One
of the following:
- true
- The job or service was executed on a remote server.
- false
- The job or service was executed on the local server.
- elapsedTime
- The time in nanoseconds that it took to execute the job or service request.
- stageInfo
- Lists execution information for each stage in the dataflow. The following information
is listed for each stage:
- stageName
- The permanent name of the stage.
- stageLabel
- The user-defined name of the stage. The stage label is shown on the canvas in Enterprise Designer.
- options
- If any options were specified at runtime, those options and their settings are listed here.
- recordsRead
- The total number of records that passed into the stage through all the stage's input ports.
- recordsWritten
- The total number of records that the stage wrote to all its output ports.
- executiontime
- The amount of time from when the stage processed its first record to when it processed its last record. This includes the time the stage was idle while waiting for data from other stages in the dataflow.
- processingtime
- The amount of time the stage spent actively processing records, not including the time it was idle while waiting for other stages in the dataflow.