Estimated Reading time: 7 minutes | Last modified: 2024-09-02, 3:26:06 pm

OpenResty Edge Configuration Mirror Backed by YAML Files

We provide a command-line tool to support updating configurations in OpenResty Edge based on local YAML file configurations, as well as exporting configurations from OpenResty Edge to YAML files. In the local configuration, application configurations are divided by application, and global configurations are divided by partition. Each local configuration corresponds to one partition and one application in OpenResty Edge. You can use version control tools like Git to manage local configurations.

Please note: This command-line tool is only supported in the Enterprise Edition of OpenResty Edge.

This document will provide detailed instructions on how to install and use this tool.

Installation

Install the dependencies required for the command-line tool
```
pip3 install --upgrade openresty-edge-sdk ruamel.yaml pyOpenSSL jinja2
```
Please ensure that the version of openresty-edge-sdk is greater than or equal to 1.2.60.

Execute the command-line tool

To view the usage of this command-line tool, execute the following command:

edge-config -h

The command output is as follows (output may vary slightly between versions):

usage: edge-config [-h] -t API_TOKEN -u ADMIN_URL [-s] [-i CONFIGS_PATH] [-e EMAIL_CONFIG_FILE] [-U USERS_CONFIG_PATH] [-F]
                  [-l LOCATION] [-d DOMAIN] [-P EXPORT_TO_PATH] [-p PARTITION_ID | -n PARTITION_NAME] [-c | -E]

Update or add OpenResty Edge configuration.

optional arguments:
  -h, --help            show this help message and exit
  -t API_TOKEN, --api-token API_TOKEN
                        specify the API Token for sending the request
  -u ADMIN_URL, --admin-url ADMIN_URL
                        specify the URL of the OpenResty Edge Admin. For example, https://admin.com:443
  -s, --disable-ssl-verify
                        turn off SSL verify for requests to access OpenResty Edge Admin
  -i CONFIGS_PATH, --configs-path CONFIGS_PATH
                        specify the path to the configuration file
  -e EMAIL_CONFIG_FILE, --email-config-file EMAIL_CONFIG_FILE
                        specify the file to the email configuration; if not specified, the email.yaml file in the configuration
                        path will be used
  -U USERS_CONFIG_PATH, --users-config-path USERS_CONFIG_PATH
                        specify the path to the users configuration; if not specified, the users/ path in the configuration
                        path will be used
  -F, --export-fake-privacy-info
                        use placeholders in place of privacy information when exporting
  -l LOCATION, --location LOCATION
                        specify the configuration name that needs to be updated, supported: page_rules, edgelang_rules,
                        upstreams, k8s_upstreams, user_variables, ip_lists, waf_whitelist, basic_auth_groups,
                        global_lua_modules, global_custom_actions, global_configs, global_page_templates, global_upstreams,
                        global_k8s_upstreams, global_ip_lists, global_basic_auth_groups, global_static_files, k8s, users, all
  -d DOMAIN, --domain DOMAIN
                        specify a domain name. When an HTTP application containing this domain exists, it will be updated;
                        otherwise, a new application will be created
  -P EXPORT_TO_PATH, --export-to-path EXPORT_TO_PATH
                        specify the storage path of the exported configuration. If not specified, a directory prefixed with
                        configs will be automatically created in the current path.
  -p PARTITION_ID, --partition-id PARTITION_ID
                        specify the id of the partition where you want to add or update the configuration
  -n PARTITION_NAME, --partition-name PARTITION_NAME
                        specify the name of the partition where you want to add or update the configuration
  -c, --cleanup         based on the location. This option allows for the cleanup of page rules, application upstreams, global
                        user variables, and resetting the access log format for partitions. It can also be used independently
                        of the location
  -E, --export          export the configuration to the specified path, which can be specified through the --export-to-path
                        option

By specifying the -p PARTITION_ID or -n PARTITION_NAME option, you can update the configuration to the specified partition.

Example:

edge-config -t 3ed00172-280e-4e43-a4ef-3c2fae5669ca -u https://192.168.1.1 -s -i /path/of/configs -n default -d test.com

This command will update the configuration to the default partition (named default, id 1), and if the test.com application doesn’t exist, it will be created.

If you encounter a No module named 'MODULE_NAME' error, use the following command to install the corresponding dependency:

pip3 install MODULE_NAME

Replace MODULE_NAME in the command with the actual module name.

Creating an API Token

Log in to the OpenResty Edge management console.
Click on the username in the upper right corner: USERNAME > API Tokens > Create API Token > Copy API Token.

Directory Structure Introduction

The command-line tool can recognize configurations with the following structure:

|-- configs/
  |-- global_lua_modules/
  |-- global_basic_auth_groups/
  |-- global_custom_actions/
  |-- global_ip_lists/
  |-- global_upstreams/
  |-- global_k8s_upstreams/
  |-- global_page_templates/
  |-- global_static_files/
  |-- k8s/
  |-- upstreams/
  |-- k8s_upstreams/
  |-- edgelang_rules/
  |-- page_rules/
  |-- basic_auth_groups/
  |-- ip_lists/
  |-- user_variables/
  |-- waf_whitelist/
  |-- users/
  |-- global_configs.yaml
  |-- app.yaml
  |-- email.yaml

The configs/ directory contains all configurations for a certain partition.

Global configurations are prefixed with global_:

global_lua_modules/: Global Lua modules, each Lua file represents a module.
global_basic_auth_groups/: Global Basic authentication user groups and user configurations.
global_custom_actions/: Global custom actions.
global_ip_lists/: Global IP lists.
global_upstreams/: Global upstreams.
global_k8s_upstreams/: Global Kubernetes upstreams.
global_page_templates/: Global page templates.
global_static_files/: Global static files.
global_configs.yaml: Includes global configurations such as global user variables, access log format, etc.

Application configurations have no prefix:

upstreams/: Upstream server information, can be split into multiple files for management.
page_rules/: Page rules, which will be added to the HTTP application related to the partition.
basic_auth_groups/: Basic authentication user groups and user configurations.
k8s_upstreams/: Kubernetes upstream server information.
edgelang_rules/: Edgelang rules at the beginning and end of the Page Rules interface.
ip_lists/: IP list information.
user_variables/: User variable information.
waf_whitelist/: WAF whitelist.
app.yaml: Application-related configurations, such as domain names, access log format.

Other configuration directories without the global prefix:

users/: Users to be added to Edge Admin, email notifications can be sent upon successful creation.
k8s/: Kubernetes cluster information.

All these configuration directories are optional, the command-line tool does not require the configuration directories to exist.

Examples of Importing Configurations to OpenResty Edge

Import all configurations to partition 1 (default) and create the application test.com when it does not exist:

edge-config -t 6cc1cf09-1a39-4c17-8ed6-c11838edac12 \
    -u https://192.168.1.1 -s -i /path/of/configs -n default -d test.com

Import page rules only to the application “test.com”:

edge-config -t 6cc1cf09-1a39-4c17-8ed6-c11838edac12 \
    -u https://192.168.1.1 -s -i /path/of/configs -n default -l page_rules -d test.com

Examples of Exporting Configurations from OpenResty Edge

Export configurations to a randomly generated directory (directory information will be output in the first line of the command-line tool):

edge-config -t 6cc1cf09-1a39-4c17-8ed6-c11838edac12 \
  -u https://192.168.1.1 -s -d test.com -E

Export configurations to a specified directory:

edge-config -t 6cc1cf09-1a39-4c17-8ed6-c11838edac12 \
  -u https://192.168.1.1 -s -d test.com -E --export-to-path /tmp/edge-configs

Examples of Deleting Configurations from OpenResty Edge

Delete the application corresponding to the domain name test.com in the default partition. If the global configuration does not enable “automatic synchronization to other partitions”, the partition configuration will also be cleaned:

edge-config -t 6cc1cf09-1a39-4c17-8ed6-c11838edac12 \
    -u https://192.168.1.1 -s -i /path/of/configs -n default -d test.com -c

Remove all page rules from the application “test.com”:

edge-config -t 6cc1cf09-1a39-4c17-8ed6-c11838edac12 \
    -u https://192.168.1.1 -s -i /path/of/configs -n default -l page_rules -d test.com -c

Remove all Lua modules from partition 1 (default) (when “Apply to all partitions” is enabled, no actual deletion will be performed as all partitions share these configurations):

edge-config -t 6cc1cf09-1a39-4c17-8ed6-c11838edac12 \
    -u https://192.168.1.1 -s -i /path/of/configs -n default -l global_lua_modules -c

Configuration Examples

For more examples, please refer to OpenResty Edge Configuration Examples Based on YAML Files.

- enable_rule: true
  actions:
    set-access-log-off: {}
    print:
      content_type: text/html
      msg: favicon.ico and robots.txt
    exit:
      code: 200
  conditions:
  - op: eq
    var: uri
    vals:
      - /favicon.ico
      - /robots.txt
  order: 1
  comment: ''
- enable_rule: true
  actions:
    - user-code:
      el: |-
        {
        my Str $transid;

        true =>
            foreign-call(module: "example", func: "go"),
            $transid = ctx-var("lm_transid"),
            $or_global_user_variable_uuid = $transid,
            done;
        }
  comment: ''

Page rules are ordered, and the order in the configuration file will be maintained when updating to Edge Admin. This example includes two page rules:

The first rule checks if the URI is /favicon.ico or /robots.txt, if so, it disables the access log and outputs specified content.
The second rule unconditionally executes a piece of Edgelang code, which sets the global user variable $or_global_user_variable_uuid.

All page rules form an array, with each page rule being an element of the array.

Error Message Examples

For configuration file errors, the command-line tool will attempt to provide hints indicating the specific line number where the error occurred.

[!] Error: invalid port in upstream upstream_name_foo: 70000, file: foo.yaml, line: 6.

There are also some other errors, such as the one below indicating that the input configuration path is incorrect, and the command-line tool cannot find the path.

[!] Error: bad configs path: /bad/path.

Additional Information

This command-line tool has two modes. When “Synchronize automatically to other partitions” is enabled for both “Global Configuration > General” and “Global Configuration > Global Lua Modules” in OpenResty Edge, to accommodate scenarios with multiple local configurations, the relevant configurations will not be reset or cleaned during update or cleanup operations. When it is not enabled, it means that each partition has its own “Global Configuration > General” and “Global Configuration > Global Lua Modules” configurations. Therefore, during update or cleanup operations, any redundant configurations in these settings will be reset or cleaned.

OpenResty Edge Configuration Mirror Backed by YAML Files

# Installation

# Creating an API Token

# Directory Structure Introduction

# Examples of Importing Configurations to OpenResty Edge

# Examples of Exporting Configurations from OpenResty Edge

# Examples of Deleting Configurations from OpenResty Edge

# Configuration Examples

# Error Message Examples

# Additional Information

Love to hear from you, Get in touch 👋

Message was sent successfully!