Entities

/api/repositories/{repository}/entities

Parameter	Type	Purpose
repository	string	The name of the Repository

Count

GET /api/repositories/(string: repository)/entities/count

Returns the number of entities in the repository.

Response Code	Reason
200	No error
500	Server error

The response contains the number of entities in the repository.

{
    "count": "23"
}

Show All

GET /api/repositories/(string: repository)/entities

Returns a list of all entities in a Repository

Paging is supported for this GET request using Standard URL Query String.

The underlying data rows can be returned in a JSON key/value pair by setting the external JSON format parameter. This allows access by different external tools that requires the JSON data in a key/value pair instead of standard array of row data with columns names defined separately.

GET /api/repositories/jupiter/entities HTTP/1.1
Accept: application/json

Response Code	Reason
200	No error

The response will be a Standard Response, with a list of the available entities in the Repository.

The columns array will contain the list of columns for each data row:

Human Name	Column Name	Description
Name	_humanised_name	The name of the entity
Ref	ENTITY_ID	Internal entity reference
Schema Name	SCHEMA_NAME	The schema name for the entity
in TSA or TSQ project	_is_in_project	Whether the entity is in a Quality or Time Series Analysis project
Attributes	_display_attributes	The number of attributes, including overflow if present
Rows	LOADED_ROWS	The number of loaded rows
Values	LOADED_VALUES	The number of distinct values loaded (across all attributes)
Row min len	RECORD_MIN_LEN	Minimum Row Length
Row max len	RECORD_MAX_LEN	Maximum Row Length
Business Rules	_business_rules	The number of business rules defined for this entity
Enabled Business Rules	_b_rules_enabled	The number of enabled business rules
Passing Business Rules	_b_rules_passed	The number of passing business rules
Failing Business Rules	_b_rules_failed	The number of failing business rules
Data Source	__DATA_SOURCE	File name of data source
Connection Data	__CONNECTION_DATA	The connection parameters for the source of the data
Encoding	__ENCODING	Encoding used when loading the data
Schema	__SCHEMA	File name of Schema Source
Permanent Keys	KEYS_PERMANENT	The number of permanent keys for this entity
Discovered Keys	KEYS_DISCOVERED	The number of discovered keys for this entity
Permanent Deps	DEPENDENCIES_PERMANENT	The number of permanent dependencies identified for this entity
Discovered Deps	DEPENDENCIES_DISCOVERED	The number of discovered dependencies identified for this entity
Permanent Joins	_joins_permanent	The number of permanent joins for this entity
Discovered Joins	_joins_discovered	The number of discovered joins for this entity
BR Compliance %	_b_rules_compliance	Measure of the entity compliance to the Business Rules
Business description	BUSINESS_DESCRIPTION	Business description
Source Type	_source_type	Source type of the loaded data
Delimiter	FIELD_DELIMITER	Field Delimiter
Quote char	QUOTE_CHAR	Quote character
Entity Type	__CREATION_TYPE	Whether the entity is Virtual, Real, Ignored or Deleted
Entity State	__STATE	The state of the entity, eg whether schema only, loaded or delete requested, etc.
Notes	note.COUNT	The number of notes for this entity
Loaded Date	_loaded_date	Date of Data Load
Loaded by user	_loaded_by_user	User who loaded the data

Each data row will contain the data for a single entity; the metadata will contain the URL to drill-down to the entity.

The standard data rows will be returned as follows:

{
    "dataRow": [
        "Cloudy",
        "5",
        "input",
        "N",
        "10",
        "50",
        "262",
        "79",
        "138",
        "0",
        "0",
        "0",
        "0",
        "input.csv",
        "Delimited",
        "cp1252",
        "input.csv",
        "0",
        "6",
        "0",
        "14",
        "0",
        "0",
        "0.000",
        "",
        "Delimited file (with labels)",
        ",",
        "",
        "Real",
        "Fully Loaded",
        "",
        "2014/11/07 16:25:17",
        "bob"
    ],
    "metadata": {
        "url": "/api/repositories/jupiter/entities/5"
    }
}

If external JSON format parameter was set then the data rows in a JSON key/value pair will be returned as follows:

{
    "Name": "Cloudy",
    "Ref": "5",
    "Schema Name": "input",
    "in TSA or TSQ project": "N",
    "Attributes": "10",
    "Rows": "50",
    "Values": "262",
    "Row min len": "79",
    "Row max len": "138",
    "Business Rules": "0",
    "Enabled Business Rules": "0",
    "Passing Business Rules": "0",
    "Failing Business Rules": "0",
    "Data Source": "input.csv",
    "Connection Data": "Delimited",
    "Encoding": "cp1252",
    "Schema": "input.csv",
    "Permanent Keys": "0",
    "Discovered Keys": "6",
    "Permanent Deps": "0",
    "Discovered Deps": "14",
    "Permanent Joins": "0",
    "Discovered Joins": "0",
    "BR Compliance %": "0.000",
    "Business description": "",
    "Source Type": "Delimited file (with labels)",
    "Delimiter": ",",
    "Quote char": "",
    "Entity Type": "Real",
    "Entity State": "Fully Loaded",
    "Notes": "",
    "Loaded Date": "2014/11/07 16:25:17",
    "Loaded by user": "bob"
    "metadata": {
        "url": "/api/repositories/jupiter/entities/5"
    }
}

If there are no entities, rows will be an empty string.

Commands

POST /api/repositories/(string: repository)/entities

Executes commands for the Entities in the repositories.

The request requires a Standard Request packet. Available commands are:

Command	Description
get	Get a filtered list of entities from the repository
create	Create a dynamic entity in the corresponding repository
preview	Get a preview of the entity
rawpreview	Get a raw preview of the entity
export	Export all or a filtered list of entities from the repository

The response is a Standard Response; the datarows array will have an element for each row that matches the filter criteria.

Get

POST /api/repositories/jupiter/entities HTTP/1.1
Content-Type: application/json

{
    "command": "get",
    "options": {
        "keypatternlist": ["1", "5"]
    }
}

Response Code	Reason
200	Success
500	Server error

The request packet must contain the command and options elements. The data element is not required.

All fields are required.

Create

POST /api/repositories/jupiter/entities HTTP/1.1
Content-Type: application/json

{
    "command": "create",
    "data": {
        "entityName": "Delimited entity",
        "entityDesc": "A Delimited entity",
        "source": "client",
        "connectInfo": {
            "type": "delimited",
            "username": "bob",
            "password": "bob",
            "sourceFile": "File1.txt",
            "schemaFile": "File1.ddl",
            "quote": "'",
            "delimiter": ",",
            "attr": "names",
            "encoding": "cp1252",
            "terminator": "crlf"
        },
        "sample": "SAMPLE {first 100}"
    }
}

Response Code	Reason
204	No error
400	Invalid data passed with request
500	Server error

The request packet must contain the command and data elements. The options element is not required, but may specify the columns to load. For example:

POST /api/repositories/jupiter/entities HTTP/1.1
Content-Type: application/json

{
    "command": "create",
    "data": {
        "entityName": "DBEntity",
        "source": "server",
        "connectInfo": {
            "connectionId": "1",
            "sourceFile": "input.csv",
            "quote": "\"",
            "delimiter": ",",
            "attr": "names",
            "encoding": "cp1252",
            "terminator": "crlf"
        }
    },
    "options": {
        "columns": ["Line_01", "Line_02", "Line_03", "Line_04", "Vin_number"]
    }
}

Note that the columnised name for each required column must be specified, not the humanised name.

Columns may include virtual columns, which are specified using this syntax: {<ATTRIBUTE_NAME> <ATTRIBUTE_NAME> {}  <ATTRIBUTE_DESCRIPTION> <ATTRIBUTE_XML_EXPRESSION> <ATTRIBUTE_SCHEMA_LENGTH>}. The third parameter is always an empty pair of braces. The recommended value for ATTRIBUTE_SCHEMA_LENGTH is 100. An example is shown below:

POST /api/repositories/jupiter/entities HTTP/1.1
Content-Type: application/json

{
    "command": "create",
    "data": {
        "entityName": "DBEntity",
        "source": "server",
        "connectInfo": {
            "connectionId": "1",
            "sourceFile": "input.csv",
            "quote": "\"",
            "delimiter": ",",
            "attr": "names",
            "encoding": "cp1252",
            "terminator": "crlf"
        }
    },
    "options": {
        "columns": ["Line_01", "Line_02", "Line_03", "Line_04", "Vin_number", "Branch", "{\"Branch plus 100\" \"Branch plus 100\" {} {This is a virtual column calculated from Branch plus 100} {<expr><text>Branch + 100</text><plus><var>Branch</var><int>100</int></plus></expr>} 100}"]
    }
}

The following table shows which fields are required in data element:

Field	Required
entityName	Y
entityDesc	N
source	Y
connectInfo	Y
sample	N

source field takes the following values:

Value	Description
client	entity has to be created using the data from client filesystem
server	entity has to be created using the data from server filesystem

sample field denotes the filtering option to be performed on the entity. It takes the following values:

Value	Description
SAMPLE all	all the samples. No filtering is performed
SAMPLE {first x}	first x records
SAMPLE {first x skip y}	Skips first y records and returns the next x records
SAMPLE {random x}	x% random records
SAMPLE {random x skip y}	Skips first y records and returns x% records randomly from the records remaining

The following table shows which fields are required in connectInfo element for delimited connection:

Field	Required for client source	Required for server source
type	Y	N
connectionId	N	Y
username	N	N
password	N	N
sourceFile	Y	Y
schemaFile	N	N
quote	N	N
delimiter	N	N
attr	Y	Y
encoding	N	N
terminator	N	N

The following table shows which fields are required in connectInfo element for cobol connection:

Field	Required for client source	Required for server source
type	Y	N
connectionId	N	Y
username	N	N
password	N	N
sourceFile	Y	Y
schemaFile	Y	Y
endian	N	N
terminator	N	N
align	N	N
redefines	N	N
upacked	N	N
encoding	N	N
national	N	N

The following table shows which fields are required in connectInfo element for trillium connection:

Field	Required for client source	Required for server source
type	Y	N
connectionId	N	Y
username	N	N
password	N	N
sourceFile	Y	Y
schemaFile	Y	Y

The following table shows which fields are required in connectInfo element for odbc, generic, oracle and db2 connections:

Field	Required
connectionId	Y
username	N
password	N
sourceFile	Y
sqlwhere	N

type denotes the type of entity to be created. This field is only applicable for entities created from client datasources. Type supports cobol, delimited, hdfs_delimited, ODBC, and trillium values. Only TSS Servers installed in a Hadoop environment(EdgeNode/Cluster mode) support the hdfs_delimited` type.

connectionId denotes the loader connection ID of the Loader Connection. That field is only applicable to entities created from loader connections.

quote field denotes the quote character used in the data. It can have the following values - NUL, `, ", ', \, /

delimiter field denotes the delimiter character used in the data. It can have the following values:

NUL, ,, |, TAB, `, /, !, \", £, $, ^, &, *, (, ), -, _, +, =, {, }, ;, :, \', @, #, ~, <, >, ?, \, ¬

attr field denotes the attribute information of the data used to create entity:

Value	Description
none	No information
names	Names on the first line
ddl	ddl file is supplied

terminator field indicates how records in the data file are terminated. The options are lf (linefeed), cr (carriage return), crlf (linefeed and carriage return), and none.

encoding field indicates the character encoding used by the data file. The following are the supported encodings:

cp1252, ascii, big5, cp1250, cp1251, cp1253, cp1254, cp1255, cp1256, cp1257, cp1258, cp437, cp737, cp775, cp850, cp852, cp855, cp857, cp860, cp861, cp862, cp863, cp864, cp865, cp866, cp869, cp874, cp932, cp936, cp949, cp950, dingbats, ebcdic, euc-cn, euc-jp, euc-kr, gb12345, gb1988, gb2312, gb2312-raw, identity, iso2022, iso2022-jp, iso2022-kr, iso8859-1, iso8859-10, iso8859-13, iso8859-14, iso8859-15, iso8859-16, iso8859-2, iso8859-3 iso8859-4, iso8859-5, iso8859-6, iso8859-7, iso8859-8, iso8859-9, jis0201, jis0208, jis0212, koi8-r, koi8-u, ksc5601, macCentEuro, macCroatian macCyrillic, macDingbats, macGreek, macIceland, macJapan, macRoman, macRomania, macThai, macTurkish, macUkraine, shiftjis, symbol, tis-620, unicode, utf-8

national field denotes the national character encoding used by the data file. It takes the same values as encoding field.

endian indicates the byte order of the data. Only big (for Big Endian) and little (for Little Endian) are supported.

align field indicates one- or two-byte data alignment. The options are one and two.

redefines field indicates how to manage REDEFINES clauses in a copybook. The options are all and first. Set it to all to account for all REDEFINES clauses in a copybook. Set it to first to ignore all REDEFINES clauses in the copybook.

upacked field indicates whether or not unsigned comp-3 fields in the copybook should be treated as comp-6. The options are 0 (for false) and 1 (for true).

sqlWhere field contains a SQL where clause for odbc connections. It is blank if no sqlWhere value is provided, and accepts conditions that filter responses, such as "Cpac < 101000000" to limit a query to records where the Cpac column value is less than 101000000.</p>

Please refer to the Repository Admin guide for further details about the connection parameters.

If you are using data files from the local filesystem, you must first call Data Upload to upload the file to the server. See the Data Upload for more details.

If the command is successful, the response will return a url link to the entity, along with the newly created entity Id as follows:

{
  "url": "/api/repositories/jupiter/entities/17",
  "entityId": "17"
}

Preview

Returns a preview of the data for any Loader Connection type, taking into account the schema sent as part of connectInfo object. This can be used before creating an entity to see the entity structure.

POST /api/repositories/jupiter/entities HTTP/1.1
Content-Type: application/json

{
    "command": "preview",
    "data": {
        "source": "client",
        "connectInfo": {
            "type": "delimited",
            "username": "bob",
            "password": "bob",
            "sourceFile": "File1.txt",
            "schemaFile": "File1.ddl",
            "quote": "'",
            "delimiter": ",",
            "attr": "names",
            "encoding": "cp1252",
            "terminator": "crlf"
        }
    }
}

Response Code	Reason
204	No error
400	Invalid data passed with request
500	Server error

The request packet must contain the command and data elements. The options element is not required, but may specify the columns to preview. For example:

POST /api/repositories/jupiter/entities HTTP/1.1
Content-Type: application/json

{
    "command": "preview",
    "data": {
        "entityName": "DBEntity",
        "source": "server",
        "connectInfo": {
            "connectionId": "1",
            "sourceFile": "input.csv",
            "quote": "\"",
            "delimiter": ",",
            "attr": "names",
            "encoding": "cp1252",
            "terminator": "crlf"
        }
    },
    "options": {
        "columns": ["Line_01", "Line_02", "Line_03", "Line_04", "Vin_number"]
    }
}

Note that the columnised name must be specified for the required columns, not the humanised name.

The following table shows which fields are required in data element:

Field	Required
source	Y
connectInfo	Y

source field takes the following values:

Value	Description
client	entity preview using the data from client/local filesystem
server	entity preview using the data from server filesystem

Refer to entity create connectInfo parameters for the connectInfo parameters. connectInfo parameters are same for entity create and preview command.

Raw Preview

This will return a raw preview of the data without any schema. This can be used before creating an entity to see the contents of the data source.

POST /api/repositories/jupiter/entities HTTP/1.1
Content-Type: application/json

{
    "command": "rawpreview",
    "data": {
        "source": "client",
        "connectInfo": {
            "type": "delimited",
            "sourceFile": "File1.txt"
        }
    }
}

Response Code	Reason
204	No error
400	Invalid data passed with request
500	Server error

The request packet must contain the command and data elements. The options element is not required.

The following table shows which fields are required in data element:

Field	Required
source	Y
connectInfo	Y

source field takes the following values:

Value	Description
client	Raw preview of the data from client filesystem
server	Raw preview of the data from server filesystem

The following table shows which fields are required in connectInfo element for delimited connection:

Field	Required for client source	Required for server source
type	Y	N
connectionId	N	Y
username	N	N
password	N	N
sourceFile	Y	Y
schemaFile	N	N
encoding	N	N

The following table shows which fields are required in connectInfo element for cobol connection:

Field	Required for client source	Required for server source
type	Y	N
connectionId	N	Y
username	N	N
password	N	N
sourceFile	Y	Y
schemaFile	Y	Y

The following table shows which fields are required in connectInfo element for trillium connection:

Field	Required for client source	Required for server source
type	Y	N
connectionId	N	Y
username	N	N
password	N	N
sourceFile	Y	Y
schemaFile	Y	Y

The following table shows which fields are required in connectInfo element for odbc, generic, oracle and db2 connections:

Field	Required
connectionId	Y
username	N
password	N
sourceFile	Y
sqlwhere	N

type denotes the type of entity to be created. That field is applicable only for entities created from client datasources. delimited, cobol and trillium are supported at the moment.

connectionId denotes the loader connection ID of the Loader Connection. That field is only applicable to entities created from loader connections.

Please refer to the Repository Admin guide for further details about the connection parameters.

If you are using data files from the local filesystem, you must first call Data Upload to upload the file to the server. See the Data Upload for more details.

Export

Some or all of the rows may be exported as a scheduled task.

POST /api/repositories/jupiter/entities HTTP/1.1
Content-Type: application/json

{
    "command": "export",
    "data": {
        "filename": "filename.csv",
        "filetype": "csv",
        "encoding": "utf-8"
    },
    "options": {
        "columns": ["_humanised_name", "ENTITY_ID"],
        "where": "Ref = 1"
    }
}

Response Code	Reason
200	No error
400	Invalid data passed with request
500	Server error

The request packet must contain the command and data elements. The options element is not required, but may specify the columns to export, and/or the rows to filter.

The following table shows which fields are required in the data element:

Field	Required
filename	Y
filetype	N
encoding	N

If filetype is not specified, it defaults to csv. If encoding is not specified, it defaults to the server native encoding.

The response will contain the URL of the scheduled task and the link to download the exported file.

{
    "job": "/api/repositories/jupiter/scheduler/1",
    "file": "/api/repositories/jupiter/download/filename.csv"
}