High Availability Support for Application Server

TigerGraph supports native HA functionality for its application server, which serves the APIs for TigerGraph’s GUI - GraphStudio and Admin Portal. The application server follows the active-active architecture, and runs on the first three nodes in a cluster by default. If one server falls offline, you can use the other servers without any loss of functionality.

When you deploy TigerGraph in a cluster with multiple replicas, it is ideal to set up load balancing to distribute network traffic evenly across the different servers. This page discusses what to do when a server dies when you haven’t set up load balancing, and the steps needed to set up load balancing for the application server.

When a server dies

When a server dies, users can proceed to the next available server within the cluster to resume the operations. For example, assuming the TigerGraph cluster has Application Server on m1 and m2. If the server on m1 dies, users can access m2 to use GraphStudio and Admin Portal.

To find out which node hosts the application server, run the gssh command in the bash terminal of any active node in the cluster. The output will show you which nodes are hosting a GUI server.

Keep in mind that any long-running operation that is currently in process when the server dies will be lost.

Configuring HA for the application server

The application server (gui) runs on the first three nodes in a cluster by default. However, you can increase the number of servers through the GUI.BasicConfig.Nodes configuration parameter.

The value of GUI.BasicConfig.Nodes is a list of nodes that run an application server:

$ gadmin config get GUI.BasicConfig.Nodes
[{"HostID":"m1","Partition":0,"Replica":1},{"HostID":"m2","Partition":0,"Replica":2},{"HostID":"m3","Partition":0,"Replica":3},{"HostID":"m4","Partition":0,"Replica":4}]

To configure the number of servers, add an JSON object to the list:

HostID is the name of the node the server runs on.
Partition must be 0.
Replica number is the number of application server that this server would be. For example, if you are adding a fourth server, then Replica should be 4.

For example, to add two more servers, add two JSON objects to the list, and use gadmin config set to set the new value. Afterwards, apply the change and restart the gui service:

$ gadmin config set GUI.BasicConfig.Nodes [{"HostID":"m1","Partition":0,"Replica":1},{"HostID":"m2","Partition":0,"Replica":2},{"HostID":"m3","Partition":0,"Replica":3},{"HostID":"m4","Partition":0,"Replica":4},{"HostID":"m5","Partition":0,"Replica":5},{"HostID":"m6","Partition":0,"Replica":6}]
$ gadmin config apply
$ gadmin restart gui

Load Balancing

When you deploy TigerGraph in a cluster with multiple replicas, it is ideal to set up load balancing to distribute network traffic evenly across the different servers.

Set up load balancing with Nginx

One possible choice for setting up load balancing is through the use of Nginx.

Here is an example Nginx configuration for the upstream and server directives:

    upstream flask_pool {
        ip_hash;
        zone flask_pool 64k;
        server 172.31.86.19:14240;
        server 172.31.88.70:14240;
        server 172.31.94.90:14240;

        keepalive 32;
    }

    server {
        listen      8000;
        server_name localhost;

        location / {
                root html;
                index index.html index.htm;
                proxy_pass http://flask_pool;
                proxy_read_timeout 3600;
                proxy_set_header Connection "";
                proxy_http_version 1.1;
                chunked_transfer_encoding off;
                proxy_buffering off;
                proxy_cache off;
        }
        error_page 500 502 503 504 /50x.html;
        location = /50x.html {
                root html;
        }
    }

The server directives should specify your nodes' addresses which you want to load balance. Since TigerGraph requires session persistence, the load balancing methods will be limited to ip_hash or hash, unless you have access to Nginx Plus, which then means any load balancing method may be used with session persistence setup: https://docs.nginx.com/nginx/admin-guide/load-balancer/http-load-balancer/#sticky

An active health check can be set on the following endpoint if using Nginx Plus:

/api/ping

Otherwise, only a passive health check is available. See Nginx documentation for more information: https://docs.nginx.com/nginx/admin-guide/load-balancer/http-health-check/

Set up AWS Elastic Load Balancer

If your applications are provisioned on AWS, another choice for load balancing is through the use of an Application Load Balancer.

To create an application load balancer, follow AWS’s guide to create an application load balancer. The following configurations apply as you follow the guide:

Configure a security group

When creating or using an existing security group in Step 3, make sure it allows requests from the load balancer to port 14240 of the instances in the target group.

Health check URL

In Step 4, set the health check URL to /api/ping

Configure targets for the target group

In Step 5, enter 14240 for the port of your instances.

Enable sticky sessions

After following the steps and creating your load balancer, enable sticky sessions in your target group.

After successfully creating your load balancer, you should now be able to access GraphStudio through the load balancer’s DNS name. The DNS name can be found under the "Description" tab of your load balancer in the Amazon EC2 console.

Set up Azure Application Gateway

If your instances are provisioned on Azure, you can set up an Application Gateway.

Follow the steps for setting up an Application Gateway outlined here:https://docs.microsoft.com/en-us/azure/application-gateway/quick-create-portal[Quickstart: Direct web traffic using the portal - Azure Application Gateway]

Some different TigerGraph specific settings are required during Application Gateway setup:

Under the section “Configuration Tab”
- For step 5, where it states to use port 80 for the backend port, use port 14240 instead.
- In the same window, enable “Cookie-based affinity”.

Create a custom probe for Application Gateway

After the Application Gateway is complete, we need to create a custom health probe in order to check the health/status of our Application Servers. You can follow the following steps outlined here:https://docs.microsoft.com/en-us/azure/application-gateway/application-gateway-create-probe-portal[Create a custom probe using the portal - Azure Application Gateway]

When filling out the health probe information, the fields below should have the following values:

Pick port from backend HTTP settings: yes

Path: /api/ping

HTTP Settings: The HTTP settings associated with the backend pool create during the Application Gateway setup

After successfully creating the Application Gateway, you should now be able to access GraphStudio from the frontend IP associated with the Application Gateway.

Set up GCP External HTTP(s) Load Balancer

If your instances are provisioned on Google Cloud, you can set up an External HTTP(s) Load Balancer:

You can follow Google’s provided steps in their documentation for setup here: Setting up an external HTTPS load balancer | Identity-Aware Proxy

When creating the instance group:

Click “Specify port name mapping”, and use 14240 for the port

When setting up the health check:

For the port, use 14240.
For the path, use /api/ping.

Lastly, we need to set up session affinity for our load balancer. This is outlined in GCP documentation here: External HTTP(S) Load Balancing overview | Google Cloud

After successfully creating the load balancer, you should now be able to access GraphStudio from the frontend IP associated with the load balancer.