Transformer stage transform types

The Transformer stage modifies field values and formatting. You can perform more than one transform on a field as long as the input and output field names are identical.

General Transforms

Construct Field

Uses values from existing fields and/or constant values to either replace field values or create a new field. For example, say you have a field named City and you want to add the phrase "City of" to the values in the City field. You would create a template like this:

City of ${City}

In the To field field, you would select the City field. This has the effect of replacing the existing values in the City field with a value constructed using the template. For example, if the value in the City field is Chicago, the new value would be City of Chicago.

Some characters must be preceded by a backslash to produce a valid template. For example, the single quote character must be preceded by a backslash like this: \'. See groovy-lang.org/syntax.html for a list of characters that must be escaped with a backslash.

Copy

Copies the value from one field to another.

Custom

Allows you to define your own transform using the Groovy language. For more information, see Creating a Custom Transform2.

For users of Spectrum Spatial , custom transforms can access spatial datasets. For more information, see the Stages section in the Spectrum Spatial Guide Spatial Stages.

Rename

Changes the name of a field. You can select from a list of field names already in the dataflow or you can type the name you want.

Status

Changes the Status field to a value of either Success or Fail. When set to Fail, an optional Description and Code may also be set.

Formatting Transforms

Case: Changes casing upper or lower case.
Mask: Applies or removes characters from a field. For more information, see Using a Mask Transform.
Pad: Adds characters to the left or right of the field value.

String Transforms

Minimize Whitespace: Removes white space at the beginning and end of the field. It also replaces any sequence of white spaces (such as multiple, consecutive spaces) to a single white space character.
Remove Substring: Removes all occurrences of a string from a field. For example, you could remove "CA" from the StateProvince field.
Substring: Copies a contiguous sequence of characters from one field to another.
Trim: Removes specified characters from the left, right, or both sides of a field. Note that this transform is case-sensitive.
Truncate: Removes a specified number of characters from the left and right sides of a field.

List Transforms

This feature helps you to create canned transformation that operate on lists, for example input from read from XML.

For defining list transformations, follow these steps:

Select a list transformation operation. Input fields appear in a tree view on the right.
Select a valid field in the tree to apply the operation on. Properties for the operation show up below the input fields tree view.
Specify the operation properties and click add. The transform gets added to the list in the parent window, that is the ‘Transformer Options’ window.

Create Field: Allows creating a field under the user selected list type field. For example if a list called Football has two clubs, Knitters and Lambs, you user can add a new club called Irons, for a total of three clubs.
Sort: Performs sorting on values present in the selected field. In a complex list, the user needs to specify the key element for sorting while in case of simple list, the sorting takes place on the elements present in the list. The user can select the sort order as either ascending or descending. In the example of Football, when the list has three clubs, the user needs to select field ‘name’ under ‘club’ to sort the clubs based on name. The current club entries list as Irons, Knitters and Lambs if sort order is ascending and descending for sort order descending. Now, if the user wants the list of players sorted, the field ‘player’ needs to be selected and sort order defined for it
Sum: Performs summation of all the values present in the selected field. The output is stored in a field specified by the user. For example, if the user wants to view the total points gained by each football club, user needs to select field ‘points’ under ‘Tournament’ and specify the output field name.
Copy: Performs the copy operation from the selected field to the field specified by the user. When user selects a field to copy, the field and all fields under it (if any) are copied to the new field specified. This operation takes place at the same level of hierarchy.
Rename: Performs the rename operation of the selected field to the new name specified by the user.

This sample XML code provides a reference to the List Transform feature:

<?xml version="1.0"?>
<sports_details>
  <sports name="football">
	<clubs>
	  <club name="Knitters">
		  <player>Samuel</player>
		  <player>Messi</player>
		  <player>kaka</player>
		  <player>Alan</player>
		  <coach>Stuart</coach>
		  <Tournament name="Football League">
			<result>won</result>
			<points>4</points>
		  </Tournament>
		  <Tournament name="UEFA">
			<result>draw</result>
			<points>2</points>
		  </Tournament>
	  </club>
	  <club name="Lambs">
		  <player>Ronaldo</player>
		  <player>Neymar</player>
		  <player>Zlatan</player>
		  <player>Mesut</player>
		  <coach>Ivan</coach>
		  <Tournament name="Airtel League">
			<result>draw</result>
			<points>2</points>
		  </Tournament>
		  <Tournament name="Champions League">
			<result>lost</result>
			<points>0</points>
		  </Tournament>
	  </club>
	  <club name="Irons">
		  <player>Scott</player>
		  <player>Paul</player>
		  <player>John</player>
		  <player>Andrew</player>
		  <coach>Jeff</coach>
		  <Tournament name="CAF">
			<result>won</result>
			<points>4</points>
		  </Tournament>
		  <Tournament name="Copa America">
			<result>won</result>
			<points>4</points>
		  </Tournament>
	  </club>
	</clubs>
  </sports>
  <sports name="badminton">
	<clubs>
	  <club name="Shuttlers">
		  <player>Saina</player>
		  <player>Viktor</player>
		  <player>Chen</player>
		  <player>Srikanth</player>
		  <coach>Jan</coach>
		  <Tournament name="Olympic Games">
			<result>won</result>
			<points>4</points>
		  </Tournament>
		  <Tournament name="Commonwealth Games">
			<result>won</result>
			<points>4</points>
		  </Tournament>
	  </club>
	  <club name="Choppers">
		  <player>Wang</player>
		  <player>Sindhu</player>
		  <player>Carolina</player>
		  <player>Li Xuerui</player>
		  <coach>Ratchanok</coach>
		  <Tournament name="World Junior">
			<result>draw</result>
			<points>2</points>
		  </Tournament>
		  <Tournament name="Uber Cup">
			<result>draw</result>
			<points>2</points>
		  </Tournament>
	  </club>
	  <club name="Lobbers">
		  <player>Nozomi</player>
		  <player>Chou</player>
		  <player>Marc</player>
		  <player>Lin</player>
		  <coach>Kevin</coach>
		  <Tournament name="World Senior">
			<result>won</result>
			<points>4</points>
		  </Tournament>
		  <Tournament name="Thomas Cup">
			<result>won</result>
			<points>4</points>
		  </Tournament>
	  </club>
	</clubs>
  </sports>
</sports_details>