Read from NoSQL DB

The Read from NoSQL DB stage reads data from a database table as input to a dataflow. The stage supports MongoDB and Couchbase database (version 5.x and above).

General Tab

Field Name Description
Connection

Select the required database connection from the dropdown list. The options displayed are based on the connections defined in Spectrum Management Console.

To add a new connection, see Connecting to NoSQL.

To modify an existing connection, select and open it from the list of connections on the Connections page of Spectrum Management Console, make the required updates, and click the Save button.

Table/View Specifies the collection or view in the database that you want to query.
Note: While the term Table/View is used in the user interface, MongoDB calls it a collection, and Couchbase calls it a view.
Schema File Click the Browse button (...) to select a JSON Schema file. This file is optional. Fields in the fields tab can be regenerated either using the schema file or database table/view.

To clear the selected file path, click Clear.

Note: Fields will always be generated using the schema file if one is selected.
Where Enter the required filter criteria, if any, using MongoDB syntax, to fetch specific records. Leave the field blank if no filter criteria is required.

This syntax is for a clause with an equal to operator:

{"<column name>" : "<filter value>"}

You can join multiple clauses using the required operators. For a list of the supported operators in the where clause, see http://docs.mongodb.org/manual/reference/operator/query/.

For example, to fetch records where the value of the column customer_name matches the value John, and the value of the column customer_age is greater than or equal to 45, enter the below:

{$and: [{"customer_name": "John"}, {$gte: ["customer_age","45"]}]}
Attention: Ensure you do not include the keyword where in this field.
Note: Currently, this field is visible only on selecting a MongoDB connection.
Ignore Absent Fields Fields defined in the schema, if not present in the actual record will not flow to the next stage if this option is selected.
Note: If you do not enable this option, fields that are not present in the database table or view, are added and processed with the value as NULL.
Preview Displays the records from the selected table.
Note: For MongoDB data sources, clicking Preview shows the filtered records, if one or more where clauses have been entered in the Where field. If no where clause has been entered, the preview displays all the records.
Note: For Couchbase data sources, clicking Preview also displays the added _id field containing the key. If the record already has an _id field, then the added _id field overwrites the pre-existing one at the time of previewing the fields.
Expand All Expands the items in the preview tree.
Collapse All Collapses the items in the preview tree.

Fields Tab

The Fields tab allows you to select the data that you want to pass to the next stage. For more information, see Defining Fields in a NoSQL Database