Recommendations

Compress data files before upload

It is recommended to compress the data files before uploading those. However, this is not a prerequisite.

Note: ZIP is the only supported data compression format.

Zipped input files are automatically unzipped before processing. The zip file must contain only the input data file with no folders within that file. The zip file cannot be password protected. .

If the input file is zipped, the output file also needs to be zipped before it is pushed to the outbound directory.

Before you zip the files, ensure that:

  • Each ZIP must have an associated trigger file.
  • Each ZIP file should only contain a single data file.
  • The base name of the trigger file, ZIP file, and compressed data file should all be the same. Only the file extensions must differ.
Example:

You want to process a customer_records.txt data file. You zip the file as customer_records.zip and upload it to the inbound directory. When the upload is complete, you upload the trigger file customer_records.trg also to the inbound directory.

Best Practice

The following guidelines must be considered when you send the data files to Spectrum OnDemand to achieve maximum performance and throughput:
  • A maximum of 10 files can be transferred to SFTP inbound directory at a time. Only when the first 10 files are successfully processed, you can send the next 10 files for processing. If you are processing multiple files at once, then each file can only contain a maximum of 100 thousand records.
  • Sorting the records by one or two fields in each file results in optimized processing.Sort the records by Postal Code, State Province, or by Country.
  • Records provided outside data subscription get credited, but results may not be accurate.