Micro-Batch Processing

Micro-batch processing is a technique where you include more than one record in a single service request. By including multiple records in a request instead of issuing separate requests for each record, you can significantly improve performance when processing a large collection of records through a service. Spectrum™ Technology Platform supports micro-batch processing for REST and SOAP web services as well for the Client SDK.

Micro-Batch Size

Using a Record ID

You may find it helpful to assign an ID to each record in a micro-batch so that you can correlate the records in the request with the records returned in the response. Use user fields to do this. For information about user fields, see The REST Interface. For information about user fields, see The SOAP Interface.

Micro-Batch Processing in REST

To perform micro-batch processing with a REST web service, include two or more records as XML or JSON in the body of the request and send the request using the POST method. For more information about sending a POST request to a Spectrum™ Technology Platform web service, see JSON POST Request and XML POST Request.

For example, this request includes two records as XML in the body of the request:

POST http://spectrum.example.com:8080/rest/ValidateAddressPOST/results.xml HTTP/1.1
Accept-Encoding: gzip,deflate
Content-Type: application/xml
Authorization: Basic YWRtaW46YWRtaW4=
Content-Length: 533
Host: config813vm0:8080
Connection: Keep-Alive
User-Agent: Apache-HttpClient/4.1.1 (java 1.5)

<ValidateAddressPOSTRequest xmlns:svc="http://www.pb.com/spectrum/services/ValidateAddressPOST">
    <svc:Input>
       <svc:Row>
          <svc:AddressLine1>3001 Summer</svc:AddressLine1>
          <svc:City>Stamford</svc:City>
          <svc:StateProvince>CT</svc:StateProvince>
       </svc:Row>
       <svc:Row>
          <svc:AddressLine1>33 west monroe</svc:AddressLine1>
          <svc:City>Chicago</svc:City>
          <svc:StateProvince>IL</svc:StateProvince>
       </svc:Row>
    </svc:Input>
</ValidateAddressPOSTRequest>

Micro-Batch Processing in SOAP

To perform micro-batch processing in a SOAP web service, include two or more records in the SOAP request. For example, this request contains two records:

POST http://spectrum.example.com:8080/soap/ValidateAddress HTTP/1.1
Accept-Encoding: gzip,deflate
Content-Type: text/xml;charset=UTF-8
SOAPAction: ""
Authorization: Basic YWRtaW46YWRtaW4=
Content-Length: 782
Host: config813vm0:8080
Connection: Keep-Alive
User-Agent: Apache-HttpClient/4.1.1 (java 1.5)

<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:val="http://www.pb.com/spectrum/services/ValidateAddress">
   <soapenv:Header/>
   <soapenv:Body>
      <val:ValidateAddressRequest>
         <val:input_port>
            <val:Address>
               <val:AddressLine1>1 N. State St.</val:AddressLine1>
               <val:City>Chicago</val:City>
               <val:StateProvince>IL</val:StateProvince>
            </val:Address>
            <val:Address>
               <val:AddressLine1>3001 summer</val:AddressLine1>
               <val:City>stamford</val:City>
               <val:StateProvince>ct</val:StateProvince>
            </val:Address>
         </val:input_port>
      </val:ValidateAddressRequest>
   </soapenv:Body>
</soapenv:Envelope>

Micro-Batch Processing in the Client API

To perform micro-batch processing in an API request to a service, send multiple data rows in the request. For example, this .NET class sends two rows in the request:

using System;
using System.Collections.Generic;
using System.Diagnostics;
using System.Linq;
using System.Net;
using System.Text;
using ConsoleApplication1.ValidateAddress_Reference;

namespace Test
{
    class Program
    {
        static void Main(string[] args)
        {
            var validateClient = new ValidateAddress {Credentials = new NetworkCredential("admin", "admin")};

            var address1 = new input_portAddress
            {
                AddressLine1 = "1825B Kramer Lane",
                AddressLine2 = "Suite 100",
                PostalCode = "78758",
                City = "Austin",
                StateProvince = "Texas"
            };

            var address2 = new input_portAddress
            {
                AddressLine1 = "100 Congress", 
                PostalCode = "78701", 
                City = "Austin", 
                StateProvince = "Texas"
            };

            var addresses = new input_portAddress[2];
            addresses[0] = address1;
            addresses[1] = address2;

            var options = new options {OutputCasing = OutputCasing.M};
            output_portAddress[] results = validateClient.CallValidateAddress(options, addresses);

            for (int i = 0; i < results.Length; i++)
            {
                System.Console.WriteLine("Record " + (i+1) + ":");
                System.Console.WriteLine("AddressLine1=" + results[i].AddressLine1);
                System.Console.WriteLine("City=" + results[i].City);
                System.Console.WriteLine("StateProvince=" + results[i].StateProvince);
                System.Console.WriteLine("PostalCode=" + results[i].PostalCode + "\n");
            }
            
            System.Console.Write("Press any key to continue...");
            System.Console.ReadKey();
        }
    }
}