Designing a Dataflow for Real-Time Revalidation
If you are using exception management in your dataflow, you can use the revalidation feature to rerun exception records through the validation process after they have been corrected in the Business Steward Portal. This enables you to determine if the change you made causes the record to process successfully in a real-time manner; you don't need to wait until the Read Exceptions batch job runs again to see the result.
The basic building blocks of a revalidation environment are:
- A job or a service that reuses or contains an exposed subflow. It must also contain an input source, the subflow stage that processes the input, a Write Exceptions stage, and an output sink for successfully processed records.
- An exposed subflow containing an Exception Monitor stage that points to a revalidation service and is configured for revalidation, including designating whether revalidated records should be reprocessed or approved.
- An exposed service that also reuses or contains the exposed subflow. It processes records that were edited, saved, and sent for revalidation in the Business Steward Portal.
Here is an example scenario that helps illustrate a revalidation
implementation:
In this example, there are three dataflows: a job, a subflow, and a service. The job
runs input data through the subflow. The subflow contains an Exception Monitor
stage, which determines if a record should be routed for manual review. Continuing
with our example, that means any records with no data in the PostalCode field would
be considered an exception and would be routed to the Write Exceptions stage; these
exceptions are what appears in the Business Steward Portal. Records with anything
else in that field would be routed to the Write to File stage.
Note: If your
dataflow is also being configured to use best of breed functionality, you will need
to manually add and expose the CollectionRecordType field in the revalidation
Exception Monitor stage/subflow and the service itself. See Write Exceptions options and
Creating a Best of Breed Record for more
information on best of breed functionality.
The exception revalidation service that you designated when configuring the Exception
Monitor stage is called when you edit one or more exception records in the Business
Steward Portal Exception Editor and click Revalidate and
Save. Like the job, the service contains the exception monitor subflow
that uses the same business logic to reprocess the record(s). If the records fail
one or more conditions set in the Exception Monitor stage, the exceptions will be
updated in the repository. If the records pass the conditions set in the Exception
Monitor stage, one of two actions will occur, depending on the selection made in the
"Action after revalidation" field:
- Reprocess records—Records will be deleted from the repository and reprocessed.
- Approve records—Records will be marked as approved and sent back to the repository.
Follow these steps to create and use a real-time revalidation scenario:
- Open or create a job or service dataflow that contains an Exception Monitor stage, an input source (such as a Read from File or Input stage), an output sink (such as a Write to File or Output stage), and a Write Exceptions stage.
- Convert the Exception Monitor stage to a subflow and map the input and output fields to match those in the initial dataflow. Be sure to include the ExceptionMetadata field for the input source as well as the output stage that populates the Write Exceptions stage in the job. Expose the subflow so it can be used by the job and service.
- Create a service that contains an Input stage, the subflow you created in step 2, an Output stage, and an output sink (such as a Write to File or Write to DB stage). Map the input and output fields to match those in the initial dataflow; be sure to include the ExceptionMetadata field for the Input stage as well as the Output stage. Expose the service so it can be used by the subflow.
- Return to the subflow and open the Configuration tab of the Exception Monitor stage. Select the revalidation service you created in step 3 and specify which action to take after revalidation. Save and expose the subflow again.
- Return to the service, where a message will appear, notifying you of changes to the subflow and saying that the service will be refreshed. Click OK, then save and expose the service again.
- Return to the initial job or service, where a message will appear, notifying you of changes to the subflow and saying that the dataflow will be refreshed. Click OK, then save the dataflow.
- Run the job. Note: Even if you have run the initial job or service before, you must run it again after creating the revalidation scenario to populate the repository with records that are eligible for revalidation. You can identify whether records in the Exception Editor are eligible for revalidation because the "Revalidate & Save" button will be active for those records.