Upgrading a Cluster with Context Graph

Before you begin this procedure, read Spectrum Server Upgrade, and make sure you have performed the prerequisites described there.

In versions of Spectrum Technology Platform earlier than 2022.1.0, each node hosts the Spectrum Technology Platform server as well as Data Hub or Context Graph models.

To upgrade a Spectrum Technology Platform cluster that runs Context Graph, complete the following tasks.

  1. Before you upgrade to Spectrum Technology Platform 2022.1.0, back up existing models and property files.
    Use the Relationship Analysis Client or the Administration Utility hub backup all CLI command to back up all Data Hub or Context Graph models from the Spectrum Technology Platform
    Back up these two property files:
    • SpectrumFolder\server\modules\hub\hub.properties
    • SpectrumFolder\server\modules\hub\db\neo4j.properties
    Important: We recommend that you create a backup before upgrading so that you can recover your flows, security settings, and other settings if an error occurs during the upgrade process.
  2. Make sure each node in the Spectrum Technology Platform cluster has been shut down.
    For more information, see Stopping a Cluster.
  3. Perform the Spectrum Technology Platform upgrade on each node in the cluster.
    Important: The installer will prompt whether to start the Spectrum Technology Platform. Make sure this option is not selected so that the server does not start automatically when the installation completes.
    For more information, see Upgrading a Cluster.
  4. Apply any updates after you complete the upgrade.
    To download and install updates for Spectrum Technology Platform 2022.1.0, see the Spectrum Update Summary pages.
  5. Start all nodes consecutively after upgrading.
    Make sure that you start node 2 within only a few seconds after starting node 1, and repeat this interval for any additional nodes.
  6. Install Neo4j Graph Database Server and configure the neo4j.conf file cluster settings for each node in a cluster configuration.
    We recommend that you install at least a 3-node Neo4j core cluster.
    For more information, see Neo4j Clustering (Neo4j Operations Manual)
  7. On each node in the Neo4j Graph Database Server cluster, complete the following steps to configure each instance to work with Context Graph.
    1. Copy the apoc jar file from Neo4jDirectory\labs\ to Neo4jDirectory\plugins\.
    2. Open the Neo4jDirectory\conf\neo4j.conf file in a code editor.
    3. Uncomment and configure the Bolt and HTTP connector port numbers as shown here: 
      # Bolt connector
dbms.connector.bolt.enabled=true
#dbms.connector.bolt.tls_level=DISABLED
dbms.connector.bolt.listen_address=:7700
dbms.connector.bolt.advertised_address=:7700

# HTTP Connector. There can be zero or one HTTP connectors.
dbms.connector.http.enabled=true
dbms.connector.http.listen_address=:7474
dbms.connector.http.advertised_address=:7474
    4. Uncomment and configure the following two property settings as follows: 
      dbms.security.procedures.unrestricted=apoc.*
dbms.security.procedures.allowlist=apoc.*
    5. Add the following setting (you can insert it in the # Miscellaneous configuration section): 
      cypher.forbid_shortestpath_common_nodes=false
    6. Optional: Add the following setting: 
      dbms.db.timezone=SYSTEM
      By default, the Neo4j Graph Database Server logs and monitoring use UTC time. This configuration specifies the local system time zone, which you may find easier to use.
    7. Optional: Uncomment and configure the following lines depending on available memory: 
      dbms.memory.heap.initial_size=512m
dbms.memory.heap.max_size=512m
      Increasing these values (for example, to 2048M and 4096M respectively) can enhance performance. For more information, see Memory recommendations (Neo4j Operations Manual).
    You may choose to configure additional properties depending on your system architecture.
  8. Start the Neo4j Graph Database Server cluster.
  9. Confirm that Neo4j Graph Database Server instances are running in the clustered configuration.
    To check the cluster status, connect to any instance in the Neo4j Browser, and enter:
    CALL dbms.cluster.overview()
    This shows information about the cluster. For more information, see Deploy a Cluster (Neo4j Operations Manual).
  10. Configure Context Graph Repository Configuration settings to enable Context Graph to communicate with the Neo4j Graph Database Server.
    1. Sign into the Spectrum Management Console and click Resources > Context Graph Repository Configuration.
    2. If a user name and password are required to connect to the Neo4j Graph Database Server, choose Basic for Authentication type, then enter the User name and Password.
    3. Optional: You would select Secured connection if the Neo4j Graph Database Server is configured to use HTTPS.
      For more information, see Use SSL to communicate with Neo4j Graph Database Server.
    4. For each Neo4j Graph Database Server instance in the cluster, click the Add address button + and specify the IP address or host name and Bolt connector port for the instance.
      Note: Certificates should be created based on the machine name where each Neo4j Graph Database Server instance is installed.
    5. Click the Test button to test the connection settings.
    6. Click Save to save the configuration settings.
  11. From one of the Spectrum cluster nodes, navigate to the SpectrumDirectory\server\modules\contextgraph\export folder, and run the exporthub.bat batch file.

    The export command creates an Output folder that contains a folder for each model. You can define the location of the output folder or use the default location. Each model folder contains the following files:

    • .csv files and import and cypher script files used to import the model into the Neo4j Graph Database Server.

      The .csv files define entities and relationships for the exported model.

    • An export.errorfile that details any exceptions that occurred while exporting the model.
    • A Summary.txt file that contains an export summary for the model.

      The export summary includes information and counts for Entities and Relationships for the model.

  12. Install a stand-alone Neo4j Graph Database Server instance on the cluster node where you ran the exporthub.bat batch file, or install the instance on a different server.
  13. Start the stand-alone Neo4j Graph Database Server instance.
  14. Navigate to the path of the output folder that resulted from running exporthub.bat.
    If the Neo4j Graph Database Server was installed on a different server, copy the output folder to that server.
  15. On the server that runs the stand-alone Neo4j instance, open the model folder and run themodelname-timestamp_import.bat script file.
    1. When prompted, enter the path to the Neo4j instance /bin folder.
      Note:

      If your data contains multiple line input fields, add the option --multiline-fields=true to the neo4j-admin import command in the modelname-timestamp_import.bat or modelname-timestamp_import.sh file.

      Example: call %neopath%/neo4j-admin import --database Artists --multiline-fields=true --verbose

  16. On the server that runs the stand-alone Neo4j instance, open the model folder and run the appropriate dbinit script file.
    SSL Certificate Run
    No None modelname-timestamp_dbinit.bat
    Yes Self-signed modelname-timestamp_dbinit_ssc.bat
    Yes Certificate authority modelname-timestamp_dbinit_s.bat
    1. When prompted, enter the user name and password for Neo4j.
      Press Enter in each case if authentication is not set up for the Neo4j instance.
    2. For the port number, enter the Bolt connector address.
      The default port setting for the Bolt connector address is 7700. This is defined on the Neo4j server in the Neo4j configuration settings file (neo4j.conf).
    3. When prompted, enter the path to the /bin folder for the Neo4j instance.
  17. On the server that runs the stand-alone Neo4j Graph Database Server instance, configure the following settings: 
    dbms.backup.enabled=true
dbms.backup.listen_address=0.0.0.0:6365
Note: Steps that follow are based on seeding a Neo4j cluster based on the Backup and Restore commands. For other methods to seed a Neo4j cluster, see Seed a cluster (Neo4j Operations Manual).
  1. Run the neo4j-admin backup command to create an online backup of the imported model on the stand-alone instance of Neo4j.

    For example, the following command backs up the artists model database on a stand-alone instance of Neo4j:

    neo4j-admin backup --from=ipAddress/hostname:6365 --backup-dir=path-to-backup-folder --database=artists
    For more information, see Back up an online database (Neo4j Operations Manual).
  2. Confirm Neo4j Graph Database Server cluster is started.
  3. Use the model backup from the stand-alone Neo4j instance, and run the neo4j-admin restore command to restore the model database on one of the core cluster nodes.

    The following command line restores the artists model to the cluster node.

    neo4j-admin restore --from=path-to-backup-folder --database=artists
    For more information, see Restore a database backup (Neo4j Operations Manual).
  4. Use the Neo4j Browser (http://localhost:7475/browser/) to connect to the cluster core node on which you ran the Restore command.
    For more information, see Neo4j Browser User Interface Guide.
  5. Select the system database and run CALL dbms.cluster.overview().
  6. From the list of nodes, find the IP address or hostname of the core node on which you ran the Restore command, and copy the node ID.
  7. From the system database run the CREATE DATABASE command to create the artists model database.

    The following example illustrates the Cypher CREATE DATABASE command to create a database model.

    CREATE DATABASE artists options {existingData: 'use', existingDataSeedInstance: 'core-node-ID'};
    The model database will replicate to the other core nodes in the cluster.
    Note: If the database is of considerable size, the execution of the command can take some time. You can reference the Neo4j logs to check the status.
  8. Verify the model database is online on all members of the cluster.
    To do this, log into the Neo4j Browser and from the system database run the Cypher SHOW DATABASES command.
After you complete this procedure, verify that you can open or query a model using Context Graph Visualization.