Collect OpenResty Edge Logs with Vector and ClickHouse

This guide describes how to collect logs from OpenResty Edge Nodes with Vector and store them centrally in ClickHouse. ClickHouse runs on a dedicated database server through Docker Compose, while Vector runs as a systemd service on each Edge Node.

Edge Node log files -> local Vector -> HTTPS -> ClickHouse

# Collection scope

After deployment, Vector writes logs to three tables in the app database:

HTTP and stream access logs must use the JSON formats provided with the deployment packages. Vector parses NGINX error logs in their default format. Logs that cannot be parsed are retained with parsed_ok set to 0, and their original contents are stored in raw_message.

Error log collection is optional. If you do not want to write error logs to ClickHouse, remove the following three complete configuration blocks from edge-vector-deployment/vector.yaml before deploying Vector:

• sources.nginx_error.
• transforms.normalize_error.
• sinks.clickhouse_error.

Remove only the Vector collection configuration. Do not delete the original error log files from the Edge Node. You can leave the unused nginx_error_logs table in ClickHouse; it remains empty and consumes no log data storage.

# Deployment prerequisites

Prepare one ClickHouse server and at least one Edge Node that has been added to Edge Admin.

This guide uses the following example values. Replace them as appropriate for your environment:

The delivery contains two packages with matching versions. The following examples use version 1.0.0:

# Extract the ClickHouse package

Download edge-clickhouse-deployment-1.0.0.tar.gz to /opt on the ClickHouse server and extract it:

cd /opt
sudo curl -O https://openresty.com/client/oredge/edge-clickhouse-deployment-1.0.0.tar.gz
sudo tar -xzf edge-clickhouse-deployment-1.0.0.tar.gz

The extracted directory has the following structure:

/opt/
└── edge-clickhouse-deployment/
    ├── .env.example
    ├── auto-deploy.sh
    ├── certs/
    ├── config.d/
    ├── docker-compose.yml
    ├── initdb/
    └── users.d/

# Extract the Vector package

Download edge-vector-deployment-1.0.0.tar.gz to /opt on every Edge Node and extract it:

cd /opt
sudo curl -O https://openresty.com/client/oredge/edge-vector-deployment-1.0.0.tar.gz
sudo tar -xzf edge-vector-deployment-1.0.0.tar.gz

The extracted directory has the following structure:

/opt/
└── edge-vector-deployment/
    ├── auto-deploy.sh
    ├── systemd/
    ├── vector.env.example
    └── vector.yaml

The extracted deployment directories do not contain version numbers. Always use /opt/edge-clickhouse-deployment and /opt/edge-vector-deployment. The two package versions must match. Stable directory names prevent version changes from affecting paths in deployment and upgrade scripts.

Verify that the time and time zone settings are correct on every host. ClickHouse stores table timestamps in UTC; query clients can convert timestamps to a display time zone as needed. Prepare a server certificate for clickhouse.example.com that is issued by a CA trusted by the clients. The certificate Subject Alternative Name must contain this hostname. Use a publicly trusted CA for a public service. If you use an internal organizational CA, install its CA certificate on every Edge Node.

# Configure Edge access logs

# Configure the HTTP access log format

1. Sign in to Edge Admin and go to Global Config > General > Log.

2. Configure the HTTP access log format in either of the following ways:

   • Modify the default main format.
   • Add a format named vector-json and make it the default access log format.

3. Replace the selected format with the following content:

   {
   "app_id": $app_id,
   "timestamp":"$time_iso8601",
   "remote_addr":"$remote_addr",
   "remote_user":"$remote_user",
   "http_host":"$http_host",
   "request":"$request",
   "status":$status,
   "body_bytes_sent":$body_bytes_sent,
   "request_time":$request_time,
   "http_referer":"$http_referer",
   "http_user_agent":"$http_user_agent",
   "upstream_addr":"$upstream_addr",
   "upstream_status":"$upstream_status",
   "upstream_connect_time":"$upstream_connect_time",
   "upstream_header_time":"$upstream_header_time",
   "upstream_response_time":"$upstream_response_time",
   "upstream_cache_status":"$upstream_cache_status",
   "pid":$pid,
   "req_id":"$req_id",
   "request_length":$request_length,
   "invalid_referer":"$invalid_referer",
   "internal_request":"$internal_request"
   }
   

4. Set Escape to json.

5. Check the HTTP applications whose logs you want to collect. Applications that use the default format automatically use this configuration. If an application explicitly selects another format, change it to main or vector-json.

6. Save and publish the configuration.

This format includes the application ID, request, status code, request latency, upstream latency, and other fields. For descriptions of the available variables, see OpenResty Edge access log variables.

Edge Admin automatically removes line breaks from a log format when it publishes the configuration. Although the JSON above is displayed across multiple lines for readability, each log entry written by the Edge Node occupies one line. The resulting file uses JSON Lines (JSONL).

# Configure the stream access log format

1. Locate the stream access log format under Global Config > General > Log.

2. Replace the original space-delimited format with the following content:

   {
   "timestamp":"$time_iso8601",
   "remote_addr":"$remote_addr",
   "protocol":"$protocol",
   "status":$status,
   "bytes_sent":$bytes_sent,
   "bytes_received":$bytes_received,
   "session_time":$session_time,
   "upstream_addr":"$upstream_addr",
   "upstream_bytes_sent":"$upstream_bytes_sent",
   "upstream_bytes_received":"$upstream_bytes_received",
   "upstream_connect_time":"$upstream_connect_time",
   "server_addr":"$server_addr",
   "server_port":$server_port
   }
   

3. Set Escape to json, then save and publish the configuration.

The JSON format generates timestamp from $time_iso8601, and Vector writes it to the ClickHouse event_time field. The remaining fields map directly to the connection status, traffic, and upstream information from the original stream access log. Edge Admin also removes line breaks from this format, so each stream log record occupies one JSONL line.

If you do not need to collect stream logs, skip this step and leave the corresponding Vector collection configuration in place. Vector produces no stream log data when no matching file exists.

# Keep the log formats consistent

The access log formats must match the table schema in the ClickHouse package of the same version. The following configurations in the two delivery packages are designed to work together:

• The HTTP and stream JSON formats in this guide define the fields emitted by Edge.
• normalize_http_access and normalize_stream_access in edge-vector-deployment/vector.yaml define field parsing and conversion.
• edge-clickhouse-deployment/initdb/001-nginx-logs.sql defines the ClickHouse column names and types.

Do not add or remove fields, or change field types, only in Edge Admin. To customize a log format, update the Vector transforms and ClickHouse schema at the same time, then verify all three configurations before publishing them.

# Match the Edge log filenames

OpenResty Edge uses stream_access.log by default and names rotated files with an underscore suffix, such as access.log_YYYYMMDD.HHMMSS. Before deploying Vector, edit edge-vector-deployment/vector.yaml and verify that the include setting of all three file sources covers both current and uncompressed rotated logs:

sources:
  nginx_http_access:
    include:
      - ${NGINX_LOG_DIR}/access.log
      - ${NGINX_LOG_DIR}/access.log_*

  nginx_stream_access:
    include:
      - ${NGINX_LOG_DIR}/stream_access.log
      - ${NGINX_LOG_DIR}/stream_access.log_*

  nginx_error:
    include:
      - ${NGINX_LOG_DIR}/*error*.log
      - ${NGINX_LOG_DIR}/*error*.log_*
      - ${NGINX_LOG_DIR}/**/*error*.log
      - ${NGINX_LOG_DIR}/**/*error*.log_*

Keep the rotated-log patterns in production. While Vector is running, it continues reading rotated files that are already open. If Vector is stopped or restarted across a rotation, patterns such as access.log_* let it use its checkpoints to resume collecting uncompressed older files. Matching only the current access.log is suitable only for real-time collection where a small amount of log loss is acceptable. It is not recommended for complete log retention.

Keep the *.gz exclusion in every source, and ensure that the collected logs do not use Gzip compression. This allows Vector to catch up on rotated files after a failure. For the default paths and rotation behavior, see Log file paths and Log rotation.

After generating test traffic on an Edge Node, verify that an HTTP access log entry is a single line of JSON:

sudo tail -n 1 /usr/local/oredge-node/logs/access.log

# Deploy ClickHouse

# Prepare the server certificate

Create certs in the ClickHouse deployment directory and place the certificate chain and private key in it:

edge-clickhouse-deployment/
└── certs/
    ├── server.crt
    └── server.key

server.crt must contain the full certificate chain. server.key must not be encrypted so that ClickHouse can start unattended. Restrict read access to the private key while ensuring that the ClickHouse process in the container can read it. Do not commit the certificate private key to version control.

# Enable ClickHouse HTTPS

The package configures edge-clickhouse-deployment/config.d/custom.xml as follows:

<clickhouse>
    <timezone>UTC</timezone>
    <max_connections>4096</max_connections>
    <listen_host>0.0.0.0</listen_host>
    <http_port>8123</http_port>
    <https_port>443</https_port>
    <tcp_port>9000</tcp_port>
    <openSSL>
        <server>
            <certificateFile>/etc/clickhouse-server/certs/server.crt</certificateFile>
            <privateKeyFile>/etc/clickhouse-server/certs/server.key</privateKeyFile>
            <loadDefaultCAFile>true</loadDefaultCAFile>
            <cacheSessions>true</cacheSessions>
            <disableProtocols>sslv2,sslv3,tlsv1,tlsv1_1</disableProtocols>
            <preferServerCiphers>true</preferServerCiphers>
        </server>
    </openSSL>
</clickhouse>

The edge-clickhouse-deployment/docker-compose.yml file mounts the certificate but does not publish host ports by default. auto-deploy.sh generates docker-compose.ports.yml from the ports supplied by the user. For example, --https-port 443 produces a configuration equivalent to:

services:
  clickhouse:
    ports:
      - "${CLICKHOUSE_BIND_ADDRESS:-0.0.0.0}:443:443"

The left side of the mapping is the user-specified host port, and the right side is the fixed container port. ClickHouse always listens on HTTPS port 443 inside the container. With the option above, the host-to-container mapping is 443:443.

Do not remove the health check from the Compose file. It checks the service through HTTP port 8123 inside the container. clickhouse-init similarly uses Native TCP port 9000 within the Compose network for initialization. These ports are published to the host only when their corresponding deployment options are explicitly supplied.

# Configure the environment variables

Run the following commands on the ClickHouse server:

cd /opt/edge-clickhouse-deployment
sudo cp .env.example .env
sudo chmod 600 .env

Edit .env with sudoedit and change at least the following settings:

CLICKHOUSE_TAG=26.3.17.56
CLICKHOUSE_DB=app
CLICKHOUSE_USER=app
CLICKHOUSE_PASSWORD=replace-with-a-strong-admin-password
CLICKHOUSE_VECTOR_PASSWORD=replace-with-a-strong-writer-password
CLICKHOUSE_OREDGE_READER_PASSWORD=replace-with-a-strong-reader-password
CLICKHOUSE_RETENTION_DAYS=365
CLICKHOUSE_BIND_ADDRESS=0.0.0.0

Use a different strong password for each account. You can generate each one with openssl rand -hex 32. The .env file contains plaintext credentials. Allow only administrators to read it, and do not commit it to version control. Public ports are no longer configured in .env; specify them as command-line options the first time you run auto-deploy.sh.

# Run the automatic deployment script

Display the supported deployment options:

cd /opt/edge-clickhouse-deployment
sudo ./auto-deploy.sh --help

The following port options are available:

If you do not specify a port option, the script publishes no ports to the host. This guide requires only HTTPS and recommends mapping the standard host HTTPS port 443 to container port 443.

Deploy with Docker:

cd /opt/edge-clickhouse-deployment
sudo ./auto-deploy.sh --https-port 443

Deploy with Podman:

cd /opt/edge-clickhouse-deployment
sudo ./auto-deploy.sh --runtime podman --https-port 443

The remaining examples use Docker. Replace docker with podman when using Podman.

Specify the port options during the initial deployment. If ClickHouse containers already exist, the script exits without changing the containers or docker-compose.ports.yml.

The script creates the clickhouse_data and clickhouse_logs external volumes, starts ClickHouse, and uses the one-shot clickhouse-init container to perform the following tasks:

• Create the three log tables and configure data retention.
• Create the vector account, which can write only to the log tables.
• Create the oredge-reader account, which can query only the log tables.

Check the container status:

sudo docker compose \
    --project-name clickhouse-deployment \
    --file docker-compose.yml \
    --file docker-compose.ports.yml \
    ps --all

An Exited (0) status for clickhouse-init is expected. The ClickHouse service must be running and healthy. If initialization fails, inspect its logs:

sudo docker compose \
    --project-name clickhouse-deployment \
    --file docker-compose.yml \
    --file docker-compose.ports.yml \
    logs clickhouse-init

# Configure network access

Allow the TCP port specified by --https-port through the firewall or security group. This guide uses 443. Do not expose the plaintext ClickHouse HTTP or Native TCP ports to the public internet. If Edge Nodes use fixed egress addresses, restrict the HTTPS port to those source addresses where possible. When source restrictions are not possible, use random strong passwords and configure connection rate limiting and anomalous-access monitoring at the server perimeter.

Verify that clickhouse.example.com resolves to the ClickHouse server, then test HTTPS connectivity from every Edge Node. The command prompts for the vector account password:

curl --user vector --data-binary 'SELECT 1' \
    https://clickhouse.example.com/

A response of 1 confirms that DNS, TLS, the port, and the credentials work. For an internal organizational CA, specify the CA certificate with --cacert /path/to/organization-ca.crt. Do not use --insecure or -k to bypass certificate verification. If --https-port is not 443, specify the same port explicitly in the URL, for example https://clickhouse.example.com:8443/.

# Deploy Vector on an Edge Node

Perform the following steps on every Edge Node whose logs you want to collect.

# Configure Vector HTTPS

The edge-vector-deployment/vector.yaml file in the package enables certificate and hostname verification for all three ClickHouse sinks:

tls:
  verify_certificate: true
  verify_hostname: true

Run the automatic deployment script and pass the hostname covered by the server certificate:

cd /opt/edge-vector-deployment
sudo ./auto-deploy.sh --host clickhouse.example.com

The script uses HTTPS port 443 by default, prompts securely for the vector account password, and automatically performs the following tasks:

• Install the openresty-vector package if it is not already installed.
• Create an unprivileged vector system account.
• Install the Vector configuration, environment file, and vector-edge.service.
• Validate the configuration and start the service.

For a non-interactive deployment, put the password on the first line of a file that only root can read:

sudo ./auto-deploy.sh --host clickhouse.example.com \
    --password-file /root/vector-password

Add --force to replace an existing managed Vector configuration:

sudo ./auto-deploy.sh --force --host clickhouse.example.com

If you specified a non-443 port with --https-port when deploying ClickHouse, pass the same port through --port when deploying Vector. For example, if ClickHouse uses --https-port 8443, run:

sudo ./auto-deploy.sh --host clickhouse.example.com --port 8443

With the recommended configuration in this guide, Vector connects to https://clickhouse.example.com:443. Do not set --host to an IP address unless the server certificate Subject Alternative Name also contains that IP. When using an internal organizational CA, add its CA certificate to the Edge Node system trust store first. Do not bypass TLS errors by disabling verification.

The primary files and data directories are:

# Grant log read permissions

Use the Edge log group or ACLs to grant access. Do not run Vector as root. The following ACL example grants access to existing logs and causes files created after log rotation to inherit the same permissions:

sudo setfacl -m u:vector:x /usr/local/oredge-node
sudo setfacl -R -m u:vector:rX /usr/local/oredge-node/logs
sudo setfacl -d -m u:vector:rX /usr/local/oredge-node/logs

If setfacl is unavailable, install the ACL utilities or grant equivalent read and directory traversal permissions through the group that owns the Edge logs.

# Check the Vector service

Reload the systemd configuration and start the service:

sudo systemctl daemon-reload
sudo systemctl enable --now vector-edge.service

On its first start, Vector reads every matching uncompressed file from the beginning. On subsequent starts, it resumes from its checkpoints. If the node already contains a large volume of historical logs, estimate the initial import first or temporarily narrow the include patterns. The HTTP log sink has a 4 GiB disk buffer, and the stream and error log sinks each have a 512 MiB buffer. Reserve sufficient space for /var/lib/vector-edge, and preserve this directory during restarts and upgrades.

Check the service status and follow its logs:

sudo systemctl status vector-edge.service
sudo journalctl -u vector-edge.service -f

After changing the configuration, reinstall the configuration file and restart the service. Do not delete the existing data directory:

cd /opt/edge-vector-deployment
sudo install -o root -g root -m 0644 vector.yaml \
    /etc/vector-edge/vector-edge.yaml
sudo systemctl restart vector-edge.service

# Check for Vector reporting errors

An active status for vector-edge.service means only that the process is running. It does not prove that logs are being written successfully to ClickHouse. Start by reviewing the service logs from the last 10 minutes:

sudo journalctl -u vector-edge.service --since "10 minutes ago" --no-pager

You can also filter for common reporting error keywords:

sudo journalctl -u vector-edge.service --since "10 minutes ago" --no-pager \
    | grep -Ei 'error|warn|failed|timeout|refused|tls|certificate|401|403|429|5[0-9]{2}|buffer'

Pay particular attention to the following messages:

The package enables a Vector API that listens only on 127.0.0.1:8686. Use vector top to observe throughput and error counts for the three ClickHouse sinks:

sudo /usr/local/openresty-vector/bin/vector top \
    --components 'clickhouse_*'

After generating test traffic, reporting has failed if ClickHouse sink error counts continue increasing, or input events continue increasing while sent events stop. A brief retry does not necessarily lose logs. As long as the disk buffer is not full, Vector resumes sending after ClickHouse recovers. Press Ctrl+C to exit vector top.

Finally, use the ClickHouse queries in the next section to verify that last_ingested_at continues to advance. The reporting path is healthy only when Vector sinks have no persistent errors, sent-event counts continue to increase, and the latest ClickHouse ingestion time remains current.

# Verify log collection

Generate several test requests through the configured HTTP or stream application, then open the read-only client on the ClickHouse server:

cd /opt/edge-clickhouse-deployment
sudo docker compose \
    --project-name clickhouse-deployment \
    --file docker-compose.yml \
    --file docker-compose.ports.yml \
    exec clickhouse clickhouse-client \
    --user oredge-reader --password --database app

Enter CLICKHOUSE_OREDGE_READER_PASSWORD from .env, then run the following queries.

Check the row count and last ingestion time for each log type:

SELECT 'http' AS type, count() AS rows, max(ingested_at) AS last_ingested_at
FROM nginx_http_access_logs
UNION ALL
SELECT 'stream', count(), max(ingested_at)
FROM nginx_stream_access_logs
UNION ALL
SELECT 'error', count(), max(ingested_at)
FROM nginx_error_logs;

View the latest HTTP requests:

SELECT
    event_time,
    app_id,
    nginx_host,
    remote_addr,
    http_host,
    request,
    status,
    request_time,
    upstream_addr,
    upstream_status
FROM nginx_http_access_logs
ORDER BY event_time DESC
LIMIT 20;

Check parsing quality:

SELECT
    nginx_host,
    parsed_ok,
    count() AS rows,
    max(ingested_at) AS last_ingested_at
FROM nginx_http_access_logs
GROUP BY nginx_host, parsed_ok
ORDER BY nginx_host, parsed_ok;

If any records have parsed_ok = 0, inspect the original logs that could not be parsed:

SELECT ingested_at, source_file, raw_message
FROM nginx_http_access_logs
WHERE parsed_ok = 0
ORDER BY ingested_at DESC
LIMIT 20;

# Capacity planning

Log length, field contents, and repetition all affect ClickHouse compression. For example, long URLs, User-Agent values, upstream addresses, and error messages increase the storage used by each log entry. Base your estimate on compressed data from representative production logs rather than only on raw log file sizes.

# Capacity formulas

Calculate HTTP, stream, and optional error log capacity separately, then add the results:

Base data capacity = logs per day × retention days × average compressed bytes per log
Planned data disk capacity = base data capacity × 2 × (1 + expected growth rate)

The recommended operational safety factor of 2 reserves space for background merges, delayed TTL deletion, data-volume fluctuations, and at least 30% free space. This factor does not include backups or replicas:

• For one full backup on the same server, add at least one more base data capacity. Store backups on a separate disk or in object storage where possible.
• For multiple ClickHouse replicas, multiply total cluster storage by the number of replicas.
• If traffic is expected to grow by 30% within one year, use 0.3 as the expected growth rate in the formula.

# Capacity example for one million logs

Without measured data, you can initially estimate 500 compressed bytes per log. One million logs then occupy approximately 0.5 GB of active ClickHouse data. After applying the operational safety factor, plan approximately 1 GB of data disk capacity for those logs.

The following estimates show how log contents affect capacity. The table uses decimal units, where 1 GB = 1,000,000,000 bytes:

These values represent only the space required by the new log data. They are not minimum disk specifications for the ClickHouse server. Reserve additional space for the operating system, container images, ClickHouse service logs, and other operational files.

# Capacity examples for one-year retention

The following examples assume that the daily total already includes HTTP, stream, and optional error logs. They use an average compressed size of 500 bytes per log, a retention period of 365 days, and an operational safety factor of 2. Additional growth, backups, and replicas are not included:

For example, if a customer generates one million logs per day and retains them for one year, start with a 365 GB data disk plan. If log volume is expected to grow by 30% within the year, adjust the plan to:

365 GB × 1.3 = 474.5 GB

Round up when purchasing storage, and use SSDs that can sustain continuous writes, background merges, and the expected query workload.

# Measure the actual size per log

Import at least one million representative production logs and wait for ClickHouse to complete the major background merges before querying system.parts. bytes_on_disk includes compressed columns, indexes, and metadata for active data parts, making it suitable for estimating actual disk usage:

SELECT
    table,
    sum(rows) AS rows,
    formatReadableSize(sum(bytes_on_disk)) AS size_on_disk,
    round(sum(bytes_on_disk) / nullIf(sum(rows), 0), 2) AS bytes_per_row
FROM system.parts
WHERE database = 'app'
  AND active
  AND table IN (
      'nginx_http_access_logs',
      'nginx_stream_access_logs',
      'nginx_error_logs'
  )
GROUP BY table
ORDER BY table;

Calculate capacity from each table’s bytes_per_row value for greater accuracy. For example, if you do not collect error logs, exclude nginx_error_logs from the calculation.

Count each type of log from the last 24 hours:

SELECT 'http' AS type, count() AS rows
FROM nginx_http_access_logs
WHERE ingested_at >= now() - INTERVAL 1 DAY
UNION ALL
SELECT 'stream', count()
FROM nginx_stream_access_logs
WHERE ingested_at >= now() - INTERVAL 1 DAY
UNION ALL
SELECT 'error', count()
FROM nginx_error_logs
WHERE ingested_at >= now() - INTERVAL 1 DAY;

Do not measure only a low-traffic day. Measure peak business days and account for promotions, attack traffic, and future growth when determining the daily log count. Re-measure regularly after the system has been running and revise the capacity forecast.

# Check disk space

Use the following query to view the total and remaining ClickHouse disk space:

SELECT
    name,
    path,
    formatReadableSize(total_space) AS total_space,
    formatReadableSize(free_space) AS free_space,
    round(free_space / total_space * 100, 2) AS free_percent
FROM system.disks;

Alert and expand capacity before free space drops below 30%. This prevents background merges or TTL cleanup from failing because of insufficient space. Monitor the clickhouse_logs volume and backup storage separately; they are not included in table capacity reported by system.parts.bytes_on_disk.

Every Edge Node also needs capacity for Vector disk buffers. When all three log types are collected, the configured buffers have a combined limit of 5 GiB. Reserve at least 6 GiB for /var/lib/vector-edge. If error log collection is disabled, you can subtract its 512 MiB buffer, but still account for a backlog while ClickHouse is temporarily unavailable.

# Change the log retention period

CLICKHOUSE_RETENTION_DAYS must be a positive integer. After changing .env on the ClickHouse server, run the initialization service again:

cd /opt/edge-clickhouse-deployment
sudo docker compose \
    --project-name clickhouse-deployment \
    --file docker-compose.yml \
    --file docker-compose.ports.yml \
    run --rm clickhouse-init

This operation updates the TTL for both new and existing log tables. ClickHouse deletes expired data during background merges, so the change does not release all space immediately.

# Troubleshooting

# Vector service does not start

View the latest errors:

sudo journalctl -u vector-edge.service -n 100 --no-pager

Check the ClickHouse hostname, HTTPS port, certificate chain, vector password, system time, and permissions on /etc/vector-edge/vector-edge.env. The environment file must be owned by root and have mode 0600.

If the logs contain certificate verify failed or unknown issuer, verify that the server sends the full certificate chain and that the Edge Node trusts the issuing CA. Do not disable verify_certificate or verify_hostname.

# Vector reports Permission denied

Verify that the vector user can traverse /usr/local/oredge-node, read the log directory and existing logs, and inherit an appropriate default ACL for files created after rotation.

# ClickHouse contains no access logs

Check the following in order:

1. The Edge application uses the configured main format or the default vector-json format, and the configuration has been published.
2. The current log file on the Edge Node contains new entries, with one JSON record per line.
3. Filenames in edge-vector-deployment/vector.yaml match the actual filenames on the Edge Node.
4. Vector logs contain no connection, authentication, parsing, or buffer errors.
5. The Edge Node can access the ClickHouse hostname and port through HTTPS and certificate verification succeeds.

# Only current logs are collected, not rotated logs

Verify that each Vector include setting covers both the current log and rotated logs with underscore suffixes, and that rotated files are not Gzip compressed. Vector uses checkpoints in /var/lib/vector-edge to avoid reading the same data again after a normal restart.

# Duplicate records appear after redeployment

Do not delete /var/lib/vector-edge. Doing so removes Vector’s read checkpoints and can cause it to import historical logs again. The ClickHouse tables use ReplacingMergeTree; records with the same sorting key are deduplicated only during background merges. This is not a replacement for Vector checkpoints.

# Recommendations for ongoing use

• Use oredge-reader, or a separate read-only account for each analytics tool, for routine queries. Do not use the vector writer account.
• Use the clickhouse-client included in the container for routine operations. Use DBeaver for graphical browsing, or the Grafana ClickHouse data source for continuous monitoring and alerting.
• Add a time range to historical log queries to avoid scanning every partition unintentionally.
• Back up the data before changing the ClickHouse image version, and preserve the clickhouse_data external volume.
• Monitor the server certificate expiration date. After renewing and replacing the certificate files, restart the ClickHouse container and test HTTPS connectivity again.

For more TLS settings, see Configure ClickHouse TLS and the Vector ClickHouse sink configuration.