Estimated Reading time: 8 minutes | Last modified: 2022-05-12, 4:09:34 pm

Upgrade OpenResty Edge

1. Upgrade OpenResty Edge Log Server

1.1 Check and record the software version before the upgrade 

# centos/redhat/fedora
rpm -q openresty-edge-log-server
# or
rpm -q openresty-edge-log-server-prod

# debian/ubuntu
dpkg -l | grep openresty-edge-log-server
# or
dpkg -l | grep openresty-edge-log-server-prod

1.2 Install timescaledb

If you are using something other than openresty-PostgreSQL, please install the timescaledb plugin yourself; If you have already installed it before, you can skip this step.

curl -O https://openresty.com/client/oredge/install-edge-postgresql-timscaledb.sh
sudo /bin/bash install-edge-postgresql-timscaledb.sh

1.3 Upgrade database 

# If this file is missing, please contact us for confirmation
# Please replace VERSION with the actual version number
/bin/bash update-log-server-db-VERSION.sh

1.4 Download upgrade script 

curl -O https://openresty.com/client/oredge/upgrade-edge-log-server.sh

If you get a message like the following:

curl: command not found

It is possible that curl is not installed on the system or that the system shell program does not contain a search path for curl.

You can use absolute paths to retry.

/usr/bin/curl -O https://openresty.com/client/oredge/upgrade-edge-log-server.sh

Your system’s path may be different, contact your webmaster or use a command such as which bash to check the path to correct the situation.

1.5 Execute the installation script 

sudo /bin/bash upgrade-edge-log-server.sh

2. Upgrade OpenResty Edge Node

We recommend starting with 1~2 Node.

  • migrate away traffic
  • Upgrade
  • Verify After it is OK, upgrade the remaining Node without migrating traffic. Also, please pay attention to the error logs during the upgrade process.

2.1 Backup Node Database

If your current OpenResty Edge Admin version is less than 1.3.0, skip this step.

  • Log in to the OpenResty Edge Admin console
  • [Gateway Clusters] - [Backup and Restore] - [Start Backup]

2.2 Check and record the software version before the upgrade 

# centos/redhat/fedora
rpm -q openresty-edge-node
# or
rpm -q openresty-edge-node-prod

# debian/ubuntu
dpkg -l | grep openresty-edge-node
# or
dpkg -l | grep openresty-edge-node-prod

2.3 Migrate traffic(optional)

To avoid disrupting operations, before a Node is upgraded, migrate the traffic on that Node to another Node that is not upgraded. This step is optional, but is highly recommended for the few Nodes that are initially upgraded. If you are using the OpenResty Edge’s DNS to control traffic, follow these steps to migrate traffic.

Access the OpenResty Edge console:

  • Enter the [Gateway Clusters].
  • [Edit] The cluster where the node is located.
  • Modify the node’s [status] to [Disable DNS, disable caching cluster].
  • [Save]

Wait for the node to no longer have traffic or only a small amount of traffic. You could confirm by checking the access log:

tail -f /usr/local/oredge-node/logs/access.log

2.4 Download and execute the upgrade script 

curl -O https://openresty.com/client/oredge/upgrade-edge-node.sh

sudo /bin/bash upgrade-edge-node.sh

3. Upgrade OpenResty Edge Admin DB

3.1 Backup Admin database

Before executing the backup command, please check if there is enough disk space.

# Please replace 127.0.0.1 with the actual IP and 5432 with the actual port.
/usr/local/openresty-postgresql12/bin/pg_dump or_edge_admin                       \
    -h 127.0.0.1 -p 5432 -U postgres                                              \
    | gzip > db_edge_admin_$(date '+%Y-%m-%d').sql.gz

3.2 Upgrade Admin database 

# If the file is missing, please contact us
# Please replace VERSION with the actual version number
/bin/bash update-admin-db-VERSION.sh

4. Upgrade OpenResty Edge Admin

Choose either this step or 5. staging upgrade Admin-web.

If you have configured double admin-web and want to staging upgrade the Admin-web service, you can jump to 5 staging upgrade Admin-web.

4.1 Check and record the software version before the upgrade 

# centos/redhat/fedora
rpm -q openresty-edge-admin
# or
rpm -q openresty-edge-admin-prod

# debian/ubuntu
dpkg -l | grep openresty-edge-admin
# or
dpkg -l | grep openresty-edge-admin-prod

4.2 Download and execute the upgrade script 

# Download upgrade script
curl -O https://openresty.com/client/oredge/upgrade-edge-admin.sh

# Execute the upgrade script
sudo /bin/bash upgrade-edge-admin.sh

4.3 Recompile

Please recompile the application in the following order:

  1. Recompile a few applications that do not contain wildcard domains with less traffic first
  2. Recompile several high-traffic applications that do not contain wildcard domains
  3. Recompile all wildcard applications
  4. Recompile all applications
  5. Recompile global rules
cd /usr/local/oredge-admin/
# recompile upgrade compatible configurations
sudo /bin/bash utils/recompile-apps.sh upgrade-config
# recompile the specified application (replace APP-ID with application ID)
sudo bash utils/recompile-apps.sh http APP-ID
# recompile all applications
sudo bash utils/recompile-apps.sh wildcard-http
sudo bash utils/recompile-apps.sh http
sudo bash utils/recompile-apps.sh http_proxy
sudo bash utils/recompile-apps.sh socks5_proxy
# recompile global rules
sudo bash utils/recompile-apps.sh global
# recompile waf rules
sudo bash utils/recompile-apps.sh waf
# recompile DNS applications
sudo bash utils/recompile-apps.sh dns
# recompile global actions
sudo bash utils/recompile-apps.sh global-action
# recompile gateway (may trigger OpenResty Edge Node reload)
sudo bash utils/recompile-apps.sh gateway

If the whole system behave normally, the upgrade is complete!

this step may take longer to compile when there are many HTTP/HTTPS applications.

sudo /bin/bash utils/recompile-apps.sh http

We recommend using another tool for concurrent compilation then. The following command will start 4 processes to compile.

sudo /bin/bash utils/parallel-recompile.sh

If it is not fast enough, you can also specify a larger concurrency, but not larger than 8, e.g.

sudo /bin/bash utils/parallel-recompile.sh 8

5. staging upgrade Admin-web

There are two prerequisites for staging upgrade of Admin-web.

  1. You have installed and configured double admin-web.
  2. Current version of Admin-web (before upgrade) is not lower than 1.3.0-1.

The architecture diagram is as follows.

Staging upgrade Admin-web architecture diagram

The key differences compared to the dual master Admin-web:

  1. Admin-B is a new version, which allows release to staging nodes with the new features, thus staging the Admin-web.
  2. Admin-A is the old version, which still provides the services of the old version. And upgrades to the dual-master state after the new version has been confirmed works well.

The specific steps are as follows.

  • Make a division of roles between the two Admin-webs, for example, Admin-A and Admin-B in the above diagram.

    Admin-A will not be upgraded during the staging period, so please make sure that the web, REST API calls, and SDK calls are using the Admin-A. Admin-B will refuse to provide web services, but will still be able to communicate with edge-node normally, so the node side will remain dual master.

  • Configure Admin-B as the staging role and upgrade to the new version

    Modify config.ini to change the role field to staging. The example configuration for the clone_admin segment in config.ini as follows.

    # Configuration file: /usr/local/oredge-admin/conf/config.ini
[clone_admin]
host = "IP-of-Admin-A"
role = "staging"

    Upgrade in the same way as above.

    # Download the upgrade script
curl -O https://openresty.com/client/oredge/upgrade-edge-admin.sh

# Execute the upgrade script.
sudo /bin/bash upgrade-edge-admin.sh

  • Configure the other Admin A role as main

    Modify config.ini to change the role field to main. An example of the configuration of the clone_admin segment of config.ini as follows

    # Configuration file: /usr/local/oredge-admin/conf/config.ini
[clone_admin]
host = "IP-of-Admin-B"
role = "main"

    Restart Admin-A.

    sudo systemctl reload oredge-admin

  • On Admin B, recompile the application and global configuration

    See 4.3 recompile for the steps.

    After completion, observe for a while and if there are problems goto Rollback I.

  • Restore the Admin B role to dual master

    Modify config.ini to change the role field to normal. An example configuration for the clone_admin segment in config.ini as follows

    # Configuration file: /usr/local/oredge-admin/conf/config.ini
[clone_admin]
host = "IP-of-Admin-A"
role = "normal"

    Restart Admin-B.

    sudo systemctl reload oredge-admin

  • Restore Admin A’s role to dual master, upgrade to the new version, and recompile the application and global configuration

    Modify config.ini to change the role field to normal. An example configuration for the ``clone_admin`' segment in config.ini as follows.

    # Configuration file: /usr/local/oredge-admin/conf/config.ini
[clone_admin]
host = "IP-of-Admin-B"
role = "normal"

    Upgrade in the same way as above.

    # Download the upgrade script
curl -O https://openresty.com/client/oredge/upgrade-edge-admin.sh

# Execute the upgrade script.
sudo /bin/bash upgrade-edge-admin.sh

    See 4.3 recompile for steps.

    If all business indicators behave properly, the upgrade is complete! You can now use Admin-A and Admin-B as dual masters again.

    If there are problems goto Rollback II

5.1 Admin-web Rollback I

  • Restore Admin-A to a dual master role, restart the service, and recompile the application and global configuration

    Modify config.ini to change the role field to normal. An example configuration for the clone_admin segment in config.ini as follows

    # Configuration file: /usr/local/oredge-admin/conf/config.ini
[clone_admin]
host = "IP-of-Admin-B"
role = "normal"

    Restart Admin-A.

    sudo systemctl reload oredge-admin

    See 4.3 recompile for recompiling steps. Note: At this point you can prioritise the recompile step that caused the error and you can restore the service as quickly as possible.

  • Restore the Admin B role to dual master and roll back the version

    Modify config.ini to change the role field to normal. An example configuration for the clone_admin segment in config.ini as follows

    # Configuration file: /usr/local/oredge-admin/conf/config.ini
[clone_admin]
host = "IP-of-Admin-B"
role = "normal"

    Downgrade in the same way as above.

    curl -O https://openresty.com/client/oredge/downgrade-edge-admin.sh
sudo /bin/bash downgrade-edge-admin.sh <OLD_PACKAGE_VERSION> [OLD_PLUS_VERSION]

# e.g.:
#  sudo /bin/bash downgrade-edge-admin.sh 1.1.0-0
#  sudo /bin/bash downgrade-edge-admin.sh 1.1.0-0 1.19.9.1.3-1

    Downgrade complete!

5.2 Admin-web Rollback II

  • Roll back Admin A and recompile

    At this point Admin A is already in dual-master mode and you only need to downgrade the version of Admin A Downgrade in the same way as above.

    curl -O https://openresty.com/client/oredge/downgrade-edge-admin.sh
sudo /bin/bash downgrade-edge-admin.sh <OLD_PACKAGE_VERSION> [OLD_PLUS_VERSION]

# e.g.:
#  sudo /bin/bash downgrade-edge-admin.sh 1.1.0-0
#  sudo /bin/bash downgrade-edge-admin.sh 1.1.0-0 1.19.9.1.3-1

    See 4.3 recompile for recompiling steps. Note: At this point you can prioritise recompiling the step that caused the error and you can restore the service as soon as possible.

  • Rolling back the Admin B version

    Downgrade in the same way as above.

    curl -O https://openresty.com/client/oredge/downgrade-edge-admin.sh
sudo /bin/bash downgrade-edge-admin.sh <OLD_PACKAGE_VERSION> [OLD_PLUS_VERSION]

# e.g.:
#  sudo /bin/bash downgrade-edge-admin.sh 1.1.0-0
#  sudo /bin/bash downgrade-edge-admin.sh 1.1.0-0 1.19.9.1.3-1

    Downgrade complete!

6. After Upgrade

After the previous steps have been performed and validated. This step can be performed to clean up database fields that are no longer needed or to add constraints to database fields.

  • Admin DB
bash /tmp/after-upgrade-admin-db-VERSION.sh

# Example: bash /tmp/after-upgrade-admin-db-202.sh

If you have any questions, please contact us.