Configuring the Options

On the Best of Breed Options: Best of Breed page, configure these options in the Settings panel:
Option Description

Group by

It specifies the field to use to create groups of records to merge into a single best of breed record, creating one best of breed record from each group. In cases where you have used a matching stage earlier in the dataflow, you should select the CollectionNumber field to use the collections created by the matching stage as the groups. However, if you want to group records by some other field, choose the field here.

For example, if you want to merge all records that have the same value in the AccountNumber field into one best of breed record, you would select AccountNumber.


If you specify a field in the Group by field, check this box to sort the records by the value in the field you choose.

Advanced Click this button to specify additional sort options. By default, the sort performance options specified in Management Console, which are the default performance options for your system, are in effect. If you want to override your system's default performance options, toggle the Override sort performance options to ON and specify the values you want in these fields:
In memory record limit
Specifies the maximum number of data rows a sorter will hold in memory before it starts paging to disk. By default, a sort of 10,000 records or less will be done in memory, and a sort of more than 10,000 records will be performed as a disk sort. The maximum limit is 100,000 records. An in-memory sort is much faster than a disk sort, so this value should be set high enough so that most of the sorts are in-memory sorts, and only large sets are written to disk.
Note: Be careful in environments where jobs are running concurrently because increasing the In memory record limit setting increases the likelihood of running out of memory.
Maximum number of temporary files
Specifies the maximum number of temporary files that may be used by a sort process. Using a larger number of temporary files can result in better performance. However, the optimal number is highly dependent on the configuration of the server running Spectrum Technology Platform. You should experiment with different settings, observing the effect on the performance of using more or fewer temporary files. To calculate the approximate number of temporary files that may be needed, use this equation:
(NumberOfRecords × 2) ÷ InMemoryRecordLimit = NumberOfTempFiles 
Note: The maximum number of temporary files cannot be more than 1,000.
Specifies that temporary files are compressed when they are written to disk.
Note: The optimal sort of performance settings depends on your server's hardware configuration. You can use this equation as a general guideline to produce good sort performance: (InMemoryRecordLimit × MaxNumberOfTempFiles ÷ 2) >= TotalNumberOfRecords
Keep original records

Check this option to retain all records in the collection along with the best of breed record. Clear this option if you want only the best of breed record.

Track master record Check this option to track the source of Best of Breed record. By enabling this option, you can select a unique input field from the drop-down to be used as a source record identifier. The selected values are then displayed in these output fields - AllSourceRecordsID and SelectedSourceRecordsID. For more information, see Output.
Note: Ensure that you select a unique input field from the drop-down list.