Batch Processing

Spectrum OnDemand batch processing gives you the ability to process many records at once from a single input data file. You upload the input data file to Spectrum OnDemand and download the resulting output data file when processing is complete.

Since your needs are unique, we customize Spectrum OnDemand batch processing to support your particular requirements. This includes selecting the appropriate services for you to use, defining the input and output file layout, and establishing naming conventions for files and fields. To learn more about batch processing or to sign up, contact your Spectrum OnDemand Account Manager.

Because batch processing is a customized service, the instructions for using your batch processing service will be provided to you based on your customized implementation. To give you a general idea of how batch processing works, here is an overview of the process and the basic requirements.

1. Upload an input data file to the inbound directory in your Spectrum OnDemand SFTP account. (You are given an SFTP account when you sign up for batch processing.) The input data file can be formatted as fixed-length, line-sequential, or delimited. The first record in the file can be a header record. Trailer records are not supported. For line-sequential and delimited files, the record separator can be a carriage return (0x0D), line feed (0x0A), or carriage return, line feed (0x0D0A). For delimited files, the field separator can be a pipe, semicolon, or comma. We recommend that you use a pipe as the delimiter since it does not appear in most data. Text values may be qualified using single or double quotes to protect against a field separator embedded in the data.
   Note: We strongly recommend that you use text qualifiers, and we recommend using double quotes as the text qualifier. If you do not use text qualifiers and the input or output data contains the delimiter character, errors may occur.

   The input data file must be encoded in ASCII, UTF-8, UTF-16, UTF-16BE, or UTF-16LE.

   We recommend that you compress the input file to a zip file, but compressing the input file is not required. If you zip the input file, the input file must not reside in a subdirectory inside the zip file, and the input file must be the only file in the zip file. If the input file is zipped, the output file will also be zipped.

   If you are performing USPS NCOA processing, the input file must be encoded as ASCII and must have at least 100 records, not counting the header record.

2. After the input data file finishes uploading, upload a trigger file. The trigger file is an empty file that indicates to Spectrum OnDemand that your file upload is complete and the file is ready for processing. The trigger file should have the same name as the data file but with a .trg file extension. For example, if the data is named customer_addresses.txt, then the trigger file should be named customer_addresses.trg.

   Files are automatically deleted from your inbound directory when they are processed.

3. When processing is complete, the output data file appears in the outbound directory in your SFTP account, along with the original trigger file. The presence of the trigger file in the outbound directory indicates that processing is complete and the output data file is ready for you to download.