On-Premises Deployment

The following is our guide for the on-premises deployment of Knowi. This guide covers key steps for connecting Docker to local databases, upgrading Knowi (Docker and non-Docker), and setting up Knowi behind an Nginx reverse proxy.

FAQs
- General Questions - Account Management - Memory - Email and Notifications - Logs and Monitoring


Below, you'll find a detailed list of all cloud9.properties settings and their definitions, followed by a comprehensive troubleshooting guide. This FAQ section is designed to address common questions and issues you may encounter during deployment.

Cloud9.properties

The following is our guide for the on-premises deployment of Knowi. Below, you'll find a detailed list of all cloud9.properties settings and their definitions.

General Configuration

Property Description
serverExternalHost=http://localhost:9090 Server public host (where Cloud9 webapp can be accessed externally)
serverInternalHost=http://localhost:9090 Server internal host (where Cloud9 webapp can be accessed internally)
apiHost=http://localhost:9090 API host where Knowi Agent connects to
#loginHost= Login URL where you can specify a custom login page.
timeZone=America/Los_Angeles Server timezone (Java compatible timezone ID)
internalDatabaseType=@db-type-name@ Relational System Database Type: Either MySQL or PostgreSQL
mongoDefaultHost=@mongo-host@ Mongo Configuration: Empty value to disable Mongo. Uncomment this if you don't want to use `java -Dcloud9.mongo.host` option. If Mongo requires authentication, the value can be in the format of :@:? where URI components should be URI encoded accordingly.
hostedAgentId=HostedAgentID Hosted Connector ID
#connectorRealtimeEnabled=false Realtime Connector: Uncomment this if you wish to disable real-time syncing from the agent.

Rabbit MQ Configuration

Property Description
messageQueueHost=localhost
messageQueuePort=5672
This part is only required if you have real-time enabled
#messageQueueUser=
#messageQueuePass=
#messageQueueSsl=
Rabbit MQ configuration

Email SMTP Server Configuration

Property Description
emailEnabled=false Enable Email
#smtpHost=
#smtpPort=
#smtpUser=
#smtpPass=
Email SMTP server configuration
#smtpTls= Set this to true to enable TLS
#smtpSslAuthentication= Set this to true to enable SSL Authentication
#supportEmail= Support Email Address

PhantomJS Configuration

Property Description
#phantomJsPath=phantomjs Location for PhantomJS for headless rendering of HTML for PDF exports. Can also be set via the -Dcloud9.phantom.js option or if PhantomJS is already included in your PATH.
#phantomCachePath= Location for PhantomJS cache
#pdfExportPath=/tmp Location to save exported PDF files
#pdfExportThreads= Number of PDF Export threads (default 1)

Knowi User Configuration

Property Description
userDefaultPlanId=10 Default plan ID (10: Unlimited)
#userDefaultCustomerId= Default customer ID for new user
#userDefaultRole= Default user role name for new user
#userDefaultGroupId= Default group ID for new user
userDataRefresh=60 Session data refresh time in minutes
ssoTokenExpiry= SSO Session Expiry (minutes)
ssoTokenSingleUse= Set to true if if you want the token to be single use (default is false)
#appHeaderPath= App header customization (can be resource name or file path)
#appFooterPath= App footer customization (can be resource name or file path)
#sslKeyStore=
#sslKeyStorePass=
SSL KeyStore settings. These can also be set via JVM system parameters `javax.net.ssl.keyStore` and `javax.net.ssl.keyStorePassword` which if set will supersede the settings here.
pwdExpirationDays=90 Users password expiration days
#reportEnabled= Enable/Disable email report (enabled by default)
#alertEnabled= Enable/Disable alert (enabled by default)

NLP Configuration

Property Description
#nlpEnabled= Enable/Disable across dataset NLP (disabled by default)
#nlpIndexerThreads= Number of NLP indexing threads
#elasticUrl= ElasticSearch URL
#elasticUsername= ElasticSearch username
#elasticPassword= ElasticSearch password
#elasticIndexPrefix= ElasticSearch index prefix to store user NLP data inside ElasticSearch (``)

Slack Integration

Property Description
#slackAppId=
#slackClientId=
#slackVerificationToken=
#slackSigningSecret=
#slackOAuthAuthorizeUrl=
Credentials for slack app integration

Management API

Property Description
#oauthProviderClientId= Management API credentials of Knowi OAuth owner for third party integrations
#oauthProviderClientSecret= Management API credentials of Knowi OAuth owner for third party integrations

Private Key for Tunneling

Property Description
#privateKeyFile=/usr/local/cloud9/.ssh/id_rsa Private key for tunneling

Performance Configuration

Property Description
#maxChunkSize= Maximum size of in-memory data chunk for processing user data (default 200000)
#maxChunkCount= Maximum number of chunks allowed for processing data (default unlimited)
#queryJoinThreads=0 Join query execution threads. These threads are used to execute each part of a single join query concurrently. Setting this to zero will disable this feature (i.e., the join parts of the query will be fetched and processed one after another by the main query thread).
#chunkSerializationMode= Serialization mode for data chunks. Valid values are FST, OOS, and NONE. FST: Fast Serialization (default) OOS: Object Output Serialization (use this if running Java 21) NONE: No serialization

URL-based Authentication for Push API

Property Description
apiKeyUrlAuthEnabled This allows turning off URL-based authentication for push API (default is TRUE)

Enhanced Session Management

Property Description
enhancedSession=TRUE Enhanced session management enables Knowi's session management when enhancedSession=TRUE. With Knowi's session management enabled, once a session is expired, a ?session expired? modal window will appear. Note: Enhanced session defaults to TRUE; however, turning enhancedSession=FALSE will default back to the existing Tomcat session management.

On Premises Troubleshooting Guide

Docker Installations

Connecting Docker to Local Databases

Issue: How to connect Docker to a local MongoDB and PostgreSQL?

Solution:

  1. Get your host machine IP:

    • Run ifconfig on your host machine and note the local network IP address.
  2. Update Docker Compose:

    • Add the extra_hosts block to the web and agent sections for both MongoDB and PostgreSQL in your docker-compose.yml file.
    • Example:
      extra_hosts:
      "mongo:10.0.0.79"
      "postgresql:10.0.0.79"

Upgrading Docker Deployment

Issue: How to upgrade Knowi Docker deployment?

Solution:

  1. Preparation:

    • Create a backup folder on your machine for file backups.
    • Stop knowi-docker-compose if it is running.
  2. Upgrade Tomcat Web App:

    • Ensure Tomcat is stopped. (Stop knowi-docker-compose)
    • Backup your current ROOT.war to the backup folder.This file should be under <knowi-docker-compose>/web/webapps/ROOT.war
    • Remove all files under <knowi-docker-compose>/web/webapps/.
    • Download and place the new WAR file in <knowi-docker-compose>/web/webapps/.
    • Restart knowi-docker-compose.
  3. Update Cloud9 Agent: Since we don't want to change any configuration of your agent, we will just replace the JAR files.

    • Stop the Cloud9 Agent. (Stop knowi-docker-compose)
    • Backup the <agent_home>/lib and <agent_home>/config folders to your backup folder.
    • Remove all JAR files under /lib: rm -rf <agent_home>/lib/*.jar.
    • Download and unzip the new Cloud9 Agent files to a temp folder and unzip.
    • Copy new JAR files to /lib/cp <new_agent_tmp_folder>/lib/*.jar <agent_home>/lib/.
    • Restart Cloud9 Agent.

Hosting Knowi Behind Nginx Reverse Proxy

Issue: How to host Knowi behind an Nginx reverse proxy in a Docker build?

Solution:

Example is for /knowi

  1. Rename war file inside <bundle>/web/webapps/ from ROOT.war to knowi.war.
  2. Update <bundle>/web/conf/cloud9/cloud9.properties to append /knowi to apiHost, serverExternalHost, and serverInternalHost.
  3. Update <bundle>/agent/config/cloud9.properties to append /knowi to apiHost.
  4. Update docker-compose.yml to replace:

./web/webapps/ROOT.war:/usr/local/tomcat/webapps/ROOT.war:ro with ./web/webapps/knowi.war:/usr/local/tomcat/webapps/knowi.war:ro

Non-Docker Installations

Upgrading Non-Docker Deployment

Issue: How to upgrade a non-Docker Knowi deployment?

Solution:

Preparation
- Create a backup folder somewhere safe on your machine.
- Stop both Tomcat Server and Cloud9 Agent (if they are running). The Agent can be stopped by either killing the process or running shutdown.sh script under <agent_home>/ folder.
- Clean up all the log files so that we can have a fresh log to debug if there is problem: - rm -rf <tomcat_home>/logs/*
- rm -rf <agent_home>/log/*
- Download the upgrade bundle and unzip. This zip contains 3 files:
- c9.license
- ROOT.war
- Cloud9AgentLibs.tar.gz

Upgrade MySQL DB Schema We are using liquibase and the schema will be upgraded automatically upon starting the server. For this reason, you should backup the "cloud9" database on MySQL first in case you need to roll back.
- Backup the cloud9 database.
- The schema will be automatically upgraded upon starting the server.

Upgrade Tomcat Web App
- Make sure that your Tomcat is stopped.
- Backup ROOT.war to the backup folder you created.
- Remove all files under <tomcat_home>/webapps/.
- Copy the new ROOT.war file from the bundle to <tomcat_home>/webapps/.
- Copy the c9.license file from the bundle to <tomcat_home>/conf/cloud9/.
- Restart Tomcat.

Update Cloud9 Agent Since we don't want to change any configuration of your agent, we will just be replacing the JAR files.
- Make sure that your Cloud9 Agent is stopped.
- Backup the <agent_home>/lib and <agent_home>/config folders to the backup folder you created.
- Remove all JAR files under <agent_home>/lib: rm -rf <agent_home>/lib/*.jar
- Copy the new Cloud9AgentLibs.tar.gz from the bundle to a temp folder and unzip.
- Copy new JAR files to <agent_home>/lib/ cp <new_agent_tmp_folder>/lib/*.jar <agent_home>/lib/
- Restart Cloud9 Agent.

Frequently Asked Questions (FAQ)

General Questions

  1. What are my deployment options?

    • Small Deployment: Tomcat + MySQL (Direct queries only)
    • Medium Deployment: Tomcat + MySQL + Mongo + Agent + RabbitMQ (Direct and scheduled queries)
    • Full Deployment: Tomcat + MySQL + Mongo + Agent + RabbitMQ + ElasticSearch (Direct and scheduled queries, Natural Language Processing)
  2. Do I need to restart Tomcat after changing cloud9.properties?

    • Yes, a restart is required for changes to take effect.
  3. How do I remove the sign-up button?

    • Set signupEnabled=false in cloud9.properties.
  4. How do I get a new c9.license for my renewal?

    • Ensure your invoice has been paid to Cloud9 Charts, Inc dba Knowi
    • Request a new license from [email protected]
  5. How do I replace my old c9.license file?

    • Rename the new license file to c9.license.
    • Replace the existing c9.license file (found in ../cloud9/c9.license)
    • Restart your tomcat server.
  6. How do I migrate user level data from one instance to another within on-premises installation?

Account Management

  1. How do I unlock an account after too many failed attempts to sign in?

    • Reset failed_login_attempts to 0 in the users table.
  2. The Ops Dashboard on my account is empty, how can I view it?

    • Ensure that the admin account's 'Datasources' section is correctly configured to point to the appropriate databases.

Memory

  1. Why is the Cloud9Agent tmp folder taking up so much space?

    • The tmp folder stores chunk files when processing large data sets that can't fit in memory. These files are temporary and should be cleaned up upon JVM exit if the agent is properly shutdown using the shutdown.sh file.
  2. How do I resolve an Out of Memory error: java.lang.OutOfMemoryError: Java heap space?

    • Check Tomcat logs for memory limit settings. Ensure the Java memory limit is set to at least 16GB (e.g., -Xmx16384m).
  3. What are the recommended memory and storage settings?

Test Environment: - Knowi Web App: 1 instance, M4 x-large or higher, 4+ CPU, 16GB+ RAM, 50GB SSD. - Knowi API Cluster: 1 instance, M4 2x-large or higher, 8+ CPU, 32GB+ RAM, 50GB SSD. - MongoDB: 1 stand-alone server, M4 2x-large or higher, 8+ CPU, 32GB+ RAM, 100GB SSD. - MySQL: 1 stand-alone server, R4 xLarge or higher, 4+ CPU, 32GB+ RAM, 50GB SSD. - Cloud9 Agent: 1 instance, M4 x-large or higher, 8+ CPU, 32GB+ RAM, 50GB SSD. - Message Queue: 1 instance, 2+ CPU, 8GB+ RAM, 20GB SSD. - ElasticSearch (Optional for NLP): 1 instance, 2+ CPU, 8GB+ RAM, 50GB SSD.

Production Environment: - Knowi Web App: 2+ instances, M4 x-large or higher, 4+ CPU, 16GB+ RAM, 50GB SSD per instance. - Knowi API Cluster: 2+ instances, M4 2x-large or higher, 8+ CPU, 32GB+ RAM, 50GB SSD per instance. - MongoDB: Cluster (refer to MySQL doc), M4 2x-large or higher, 8+ CPU, 32GB+ RAM, 400GB SSD. - MySQL: 1+ clusters (refer to MongoDB doc), R4 xLarge or higher, 4+ CPU, 32GB+ RAM, 50GB SSD. - Cloud9 Agent: 2+ instances, M4 x-large or higher, 8+ CPU, 32GB+ RAM, 50GB SSD. - Load Balancers (UI and API): ELB equivalent, 2+ instances. - Message Queue: 2+ instances, R4 large or higher, 2+ CPU, 8GB+ RAM, 50GB SSD. - ElasticSearch (Optional for NLP): 1 instance, 2+ CPU, 8GB+ RAM, 50GB SSD.

Email and Notifications

  1. Unable to send emails after entering SMTP credentials in c9.properties. What should I do?

    • Verify that the credentials under #Email in c9.properties are accurate and that emailEnabled=true.
    • Test with Telnet:

      Telnet is a command-line tool that can help diagnose SMTP connection problems. Open a command prompt or terminal and type: telnet 25 (replace with the actual IP address of the SMTP server). 25 refers to the port number. If the connection is successful, you should see a response from the server, indicating that you have connected to the SMTP port. If the connection fails or times out, it suggests a connectivity issue.

  2. How do I set up a notification if a query has not been run on the agent?

    • Set up a data update alert on the dataset you want to monitor.

Logs and Monitoring

  1. Where can I locate my log files?

    • Tomcat logs: <tomcat_home>/logs (system events and errors)
    • Agent logs: <agent_home>/logs (query errors and executions)
  2. How do I adjust my log4j2.xml to direct logs to my own logging platform?

    • Edit /usr/local/webapps/ROOT/WEB-INF/classes/log4j2.xml or replace ROOT with your webapp's name if it has been renamed.
  3. How do I resolve the MYSQL CLIENT_AUTH_PLUGIN Required error?

    • Check for MySQL version / connector JAR mismatch.
    • Check for multiple JAR versions in the /lib folder.
    • Restart Tomcat after making changes.