Flat and Hierarchical Data

Flat Data

Flat data consists of records, one on each line, and fields in each record. Fields are delimited by a specific character or positioned in a defined location on the line. For example, this is flat data with comma-delimited fields:

Sam,43,United States
Jeff,32,Canada
Mary,61,Ireland

To read flat data into a flow, you can use the Read from File, Read from DB, or Input stages. To write flat data output from a flow, you can use the Write to File, Write to DB, or Output stages.

Hierarchical Data

Hierarchical data is a tree-like structure with data elements that have parent-child relationships. Spectrum Technology Platform can read and write hierarchical data in XML and Variable Format File format. For example, this shows hierarchical data in XML:

<customers>
	<customer>
		<name>Sam</name>
		<age>43</age>
		<country>United States</country>
	</customer>
	<customer>
		<name>Jeff</name>
		<age>32</age>
		<country>Canada</country>
	</customer>
	<customer>
		<name>Mary</name>
		<age>61</age>
		<country>Ireland</country>
	</customer>
</customers>

This example shows a structure where <customer> represents a record and each record consists of simple XML elements (<name>, <age>, and <country>).

Converting Data

There are many cases where you might need to convert data from flat to hierarchical, or from hierarchical to flat. For example, you may have data flow input in hierarchical format but want the data flow to output flat data. You may also need to convert flat input data to hierarchical data for certain stages (especially stages in Spectrum Spatial) then convert the data back to flat data for output.

To convert data from flat to hierarchical you can use:

• The Process List tool
• The Aggregator stage in a flow

To convert data from hierarchical to flat use the Splitter stage.