OpenResty XRay Standard Analyzers

OpenResty XRay provides standard analyzers for the following application types or stack stacks (the list is still growing fast).

# All

OpenResty XRay provides the following tools for analyzing All applications:

# CPU

• c-on-cpu: C-Land CPU Flame Graph
• cpu-hogs: cpu hogs of the system
• epoll-level-event-fgraph: Epoll Level Trigger Event Flame Graph
• epoll-loop-blocking-distr: Epoll Loop Blocking Latency Distribution
• epoll-wait-ret-distr: Return Value Distribution of epoll_wait
• epoll-wait-timers: Wait timers for epoll
• epoll-wait-timers-fgraph: Wait timers flamegraph for epoll
• func-call-rate: Function Call Rate
• image-inspect: inspect image meta data by process ID
• kernel-on-cpu: Kernel-Land on-CPU Flame Graph
• page-fault-by-pid: Page fault order by process ID
• pagefault-fgraph: Page Fault Flame Graph
• process-ctx-sw-stats: Process Context Switches Statistics
• sys-load-avg-by-app: System Load Average By Application
• sys-softirq-stats: System Softirq Statistics
• sysctl: Run sysctl -a to get all sysctl parameters

# Disk I/O

• bench-io-create-files: benchmark of creating and writing into the files
• diskstat: Virtual File System Read Write By Disk Name
• disktop-by-proc: Virtual File System Read Write N Process
• file-system-fgraph: File System Flame Graph
• vfs-read-fgraph: Virtual File System Read/Sendfile Flame Graph
• vfs-read-latency-fgraph: Virtual File System Read/Sendfile Latency Flame Graph
• vfs-read-latency-top-files: Virtual File System Read Latency Top N files (including sendfile)
• vfs-read-top-files: Virtual File System Read Top N files (including sendfile)
• vfs-read-volume-fgraph: Virtual File System Read/Sendfile Volume Flame Graph
• vfs-write-fgraph: Virtual File System Write Flame Graph
• vfs-write-latency-fgraph: Virtual File System Write Latency Flame Graph
• vfs-write-latency-top-files: Virtual File System Write Latency Top N files
• vfs-write-top-files: Virtual File System Write Top N files
• vfs-write-volume-fgraph: Virtual File System Write Volume Flame Graph

# Latency

• epoll-wait-overrun-time-distr: Epoll Wait Overrun Time Distribution
• func-latency-distr: Function Latency Distribution
• socket-send-recv-latency-fgraph: Socket Send/Recv Message Latency Flame Graph

# Memory

• c-alloc-fgraph: C Memory Allocation Flame Graph
• c-count-alloc-free: C Application-Level Memory API Calls
• c-memory: C Application-Level Memory Usage Breakdown
• c-memory-leak-fgraph: C-Land Memory Leak Flame Graphs
• kmalloc-leak-fgraph: kmalloc leakage Flame Graph
• libc-chunks: Distributions of Memory Chunk Sizes of the Memory Allocator
• libc-dump-mem: Dump the used memory allocated by malloc and related functions
• libc-mallinfo: Information about memory allocations performed by malloc and related functions
• mmap-leaks: Mmap Leak Backtrace Sampling Tools for Leak Flame Graphs
• process-memory: Virtual Memory Usage Breakdown
• tcmalloc-bin-distr: Distribution of Memory Usage across tcmalloc Size Classes and Page Heap
• thread-stack-loaded: Loaded stack memory used by all the threads

# Miscellaneous

• gen-core-dump: Generate the core dump file via gcore and optionally bundle it with related ELF files
• generic-core-dump-analysis: Generic Core Dump Analysis
• get-dlopen-libs: Get libraries loaded by dlopen
• proc-reap: Reap the snapshot process
• proc-snap: Create a snapshot of the process (only for expert)
• signal-watcher: Sent or Received signals
• syscall-execve: fork and execute commands
• syscall-write-stat: Statistic of errno for write/writev/send syscall

# Network I/O

• kernel-dropwatch: Packet Drop Flame Graph
• socket-leak-fgraph: Socket Leakage Flame Graph
• socket-send-recv-message: Socket Send/Recv Message Flame Graph
• socket-send-recv-volume-fgraph: Socket Send/Recv Message Volume Flame Graph
• sys-nf-conntrack: current config of nf-conntrack and its recommend value
• tcp-reset-fgraph: TCP Reset Flame Graph

# off-CPU

• c-off-cpu: C-Land off-CPU Flame Graph

# APISIX

OpenResty XRay provides the following tools for analyzing APISIX applications:

# CPU

• lj-add-timer-lua-fgraph: ngx.timer Add Timer Flamegraph
• lj-c-on-cpu: C-Land CPU Flame Graph for LuaJIT
• lj-dump-traces: Dump LuaJIT Traces
• lj-excep-lua-fgraph: Lua Exception Flame Graph
• lj-func-events: Trace LuaJIT function call entry events
• lj-gc-step-calls: Number of LuaJIT GC function calls per second
• lj-jit-state: Check The LuaJIT JIT State: on/off
• lj-live-coroutines: dump the bts of the living Lua coroutines
• lj-lua-full-bt: Dump Lua Full Backtrace including the variables in the stack
• lj-lua-newcdata: Lua-Land New CData Creation Flame Graph Sampling Tool for LuaJIT Applications.
• lj-lua-newfunc: Lua-Land New Function Creation Flame Graph sampling Tool for LuaJIT Applications.
• lj-lua-newgco: LuaJIT GC Object Allocation Flame Graph
• lj-lua-newstr: LuaJIT String Objects Allocation Flame Graph
• lj-lua-newtab: LuaJIT Table Objects Allocation Flame Graph
• lj-lua-newthread: LuaJIT Thread Objects Allocation Flame Graph
• lj-lua-on-cpu: Lua-Land CPU Flame Graph
• lj-lua-tab-resize: LuaJIT Table resize Flame Graph
• lj-no-pcre-jit-lua-fgraph: Lua Land Flame Graph for function path that PCRE jit is disabled
• lj-slowest-ngx-re: Slowest find/match/split Regular Expression of ngx.re
• lj-slowest-str-gmatch: Slowest Lua string.gmatch
• lj-slowest-str-gsub: Slowest String Global Substitution For Lua’s string.gsub
• lj-slowest-str-match-find: Slowest String Match/Find
• lj-str-find-no-plain-fgraph: Lua string.find Searching Plain String Without Plain Mode Set
• lj-trace-abort-events-fgraph: Lua Land Trace Abort Events Flame Graph
• lj-trace-stats: LuaJIT VM Trace Event Statistics
• lj-vm-states: LuaJIT VM States Distribution
• ngx-access-log-buffer-size: Nginx Access Log Buffer Size Configuration
• ngx-brotli-compression-level: Nginx Brotli Compression Level Statistics
• ngx-brotli-compression-types: Nginx Brotli Compression Types Statistics
• ngx-cpu-hottest-hosts: Showing the top 10 hostnames consuming the most CPU resources
• ngx-cpu-hottest-uris: Showing the top 10 URIs order by request count
• ngx-epoll-event-stats: Statistics of Epoll Events Received
• ngx-epoll-wait-timers: Tracking the Nginx-Land Epoll Wait Timeout Values
• ngx-gzip-compression-level: Nginx GZIP Compression Level Statistics
• ngx-gzip-compression-types: Nginx GZIP Compression Types Statistics
• ngx-lua-code-cache: Nginx Lua Code Cache Requests
• ngx-open-file-cache-conf: Nginx Open File Cache Configuration
• ngx-pcre-compile-lua-fgraph: Openresty New PCRE Compilation Lua Land Flame Graph
• ngx-shdict-hit-rate: Nginx Shared Dict Hit Rate
• ngx-slowest-config-re: Slowest Regular Expression in Nginx Rewrite or Find Config
• ngx-stats: Nginx Statistics from nginx_status
• ngx-timer-delay-distr: Timeout Value Distribution of Nginx Timer
• ngx-zstd-compression-level: Nginx ZStandard Compression Level Statistics
• ngx-zstd-compression-types: Nginx ZStandard Compression Types Statistics
• openresty-compiled-regex-cache: OpenResty Compiled Regex Cache Statistics
• openresty-cosocket-handshake: Cosocket SSL Handshake Flame Graph
• openresty-new-timer-fgraph: Openresty New Timer Flamegraph
• openresty-running-timers-fgraph: Openresty Running Timer Flamegraph

# Disk I/O

• lj-read-fgraph: Lua-Land Virtual File System Read/Sendfile Flame Graph
• lj-read-latency-fgraph: Lua-Land Virtual File System Read/Sendfile Latency Flame Graph
• lj-read-volume-fgraph: Lua-Land Virtual File System Read/Sendfile Volume Flame Graph
• lj-write-fgraph: Lua-Land Virtual File System Write Flame Graph
• lj-write-latency-fgraph: Lua-Land Virtual File System Write Latency Flame Graph
• lj-write-volume-fgraph: Lua-Land Virtual File System Write Volume Flame Graph

# Error

• lj-lua-ngx-print-error: ngx.print and ngx.say error Flame Graph

# Latency

• ngx-err-log-files: Nginx Error Log Files
• ngx-http-access-time: Nginx HTTP Access Phase Latency
• ngx-http-rewrite-time: Nginx HTTP Rewrite Phase Latency
• ngx-http-upstream-time: Nginx HTTP Upstream Latency
• ngx-https-handshake-time: Nginx HTTPs handshake - useless
• ngx-read-req-body-time: Nginx Request Body Reading Time Distribution
• ngx-req-latency: Nginx Request Latency
• ngx-req-latency-advance: Nginx Request Latency with Lua Profiling
• ngx-req-latency-distr: HTTP Request Latency Distribution
• ngx-req-lj-yield-count: Nginx Request LuaJIT Yield Times Flame Graph
• ngx-req-lj-yield-latency: Nginx Request LuaJIT Yield Latency Flame Graph
• ngx-req-lj-yield-time: Nginx Request LuaJIT Yield Latency
• ngx-tls-handshake-latency: Nginx SSL/TLS handshake Latency
• openresty-coro-timeout-packets: Nginx Request Latency
• sema-wait-fgraph: Semaphore wait Calls

# Memory

• lj-alloc-stats: Statistics for LuaJIT’s builtin Allocator (Directly allocated memory blocks larger than 128K will not be counted)
• lj-c-memory-leak-fgraph: Lua-Land Memory Leak Flame Graphs
• lj-dump-cdata-fin: Dump LuaJIT CDATA finalizer
• lj-dump-loaded-mods: Showing the loaded modules of the LuaJIT VM
• lj-err-mem: Luajit Memory Exception
• lj-free-stats: Statistics of free chunks for LuaJIT’s builtin Allocator
• lj-gco-cdata-fin: CData Object Statistics by Finalizer
• lj-gco-cdata-stats: CData Object Statistics by CData Type
• lj-gco-ref: GC Object Reference Flame Graph
• lj-gco-stat: Statistics for LuaJIT GC Objects
• lj-lua-newgco-size: LuaJIT GC Object Allocation Size Flame Graph
• lj-mem-realloc-distr: Lua Memory Realloc Size Distribution
• lj-mem-realloc-fgraph: Lua Memory Realloc Size Flamegraphs
• lj-tab-chain-loops: Find loops in LuaJIT table node chains.
• lj-tab-unreachable-node: Find unreachable nodes in the lua tables.
• ngx-loaded-modules: Nginx Loaded Modules
• ngx-mem-pools: Nginx Memory Pool Statistics
• ngx-shdict-stores: Nginx Shdict Stores Statistics
• ngx-shm-lru: Nginx Share Memory LRU Statistics
• ngx-shm-slabs: Slab Memory Usage in Nginx Shared Memory
• ngx-shm-tree-height: Compute the RBTree Height of Nginx Shared Memory Zones.
• resty-memory: Application-Level Memory Usage Breakdown

# Miscellaneous

• collect-luajit-ffnames: Collect LuaJIT ffnames
• lj-gc-state: LuaJIT GC configuration and runtime state
• lj-lua-new-timer-errors: Nginx Lua Tracks Timer Creation Errors
• ngx-config: Configurations for Nginx
• ngx-config-servers: Dump The Name of The Http Servers
• ngx-dump-timers: Dump nginx timers in the rbtree
• ngx-graceful-exits: Nginx Graceful Exit Event
• ngx-lua-sleep: Nginx Lua Sleep Calls Distribution
• ngx-lua-sleep-fgraph: Nginx Lua Sleep Flame Graph
• ngx-worker-id: Nginx Worker ID
• openresty-core-dump-analysis: OpenResty Core Dump Analysis

# Network I/O

• lj-lua-ngx-exit-fgraph: Lua-Land Nginx Request Status Code Flame Graph
• lj-lua-socket-close-fgraph: Lua-Land close socket Flame Graph
• ngx-downstream-keepalive-stats: HTTP Downstream Request Connection Keepalive Statistics
• ngx-downstream-ssl-session-reused: Nginx Downstream SSL/TLS Session Reused Statistics
• ngx-dump-cur-req: Dump Nginx Request
• ngx-dump-dns: Dump Nginx DNS configuration and domain caches
• ngx-dump-req: Dump Nginx Request
• ngx-keepalive-statistics: Request/Upstream Connection Keep-alive Statistics
• ngx-proxy-cache-status: Proxy Cache Status Distribution
• ngx-read-req-body-packets: Capture the packets of the HTTP request that the read body time is greater than the threshold
• ngx-req-count: Count Nginx Subrequests and Main Requests
• ngx-req-counts-by-hosts: Showing the top 10 hostnames order by request counts
• ngx-req-distribution: Nginx Traffic Distribution By Calls of Finalize Request
• ngx-req-err-packets: IP Packets of Specified HTTP Errors
• ngx-req-size-by-hosts: Showing the top 10 hostnames order by request size
• ngx-req-status-code: Request Status Code Per Second
• ngx-rps: Nginx RPS Statistics
• ngx-ssl-server-conf: Sample the Server Configurations by Requests
• ngx-ssl-session-ticket-key: Nginx SSL Session Ticket Key Configuration
• ngx-status-code-fgraph: Nginx Request Status Code Flame Graph
• ngx-tls-handshake-info: SSL/TLS Session handshake information Statistics
• ngx-tls-session-resumption: Downstream SSL/TLS Session Resumption Statistics
• ngx-top-latency-reqs: IP packets of top latency HTTP requests
• ngx-upstream-err-packets: Capture the upstream packets that have specific error code
• ngx-upstream-no-keepalive: capture the packets that have no HTTP upstream ‘keepalive’ configuration
• ngx-upstream-ssl-session-reused: Upstream SSL/TLS Session Reused Statistics
• openresty-cosocket-connect: OpenResty Cosocket Connect Operation Flame Graph
• openresty-cosocket-err-fgraph: OpenResty Cosocket Error Flame Graph
• openresty-cosocket-err-packets: IP Packets of OpenResty Cosocket Which has Errors
• openresty-cosocket-no-keepalive: OpenResty Cosocket Without Keepalive Flame Graph
• openresty-cosocket-pool: OpenResty Cosocket Pool Statistics
• openresty-cosocket-timeout: OpenResty Cosocket Timeout Distribution
• openresty-lua-phase-cpu-time: OpenResty Lua Phase CPU time statistics
• openresty-lua-phase-wall-time: OpenResty Lua Phase wall time statistics
• socket-close-fgraph: close socket Flame Graph

# off-CPU

• lj-c-off-cpu: C-Land off-CPU Flame Graph for LuaJIT
• lj-lua-off-cpu: Lua-Land off-CPU Flame Graph
• ngx-err-log-lvl-distr: Nginx Error Log Level Distribution

# C/C++

OpenResty XRay provides the following tools for analyzing C/C++ applications:

# CPU

• c-on-cpu: C-Land CPU Flame Graph

# Dlang

OpenResty XRay provides the following tools for analyzing Dlang applications:

# Memory

• d-newgco: D GC Object Allocation Flame Graph
• d-newgco-size: D GC Object Allocation Size Flame Graph

# Envoy

OpenResty XRay provides the following tools for analyzing Envoy applications:

# CPU

• c-on-cpu: C-Land CPU Flame Graph
• envoy-lua-on-cpu: Envoy Lua-Land on-CPU Flame Graph

# off-CPU

• c-off-cpu: C-Land off-CPU Flame Graph
• envoy-lua-off-cpu: Envoy Lua-Land off-CPU Flame Graph

# Erlang

OpenResty XRay provides the following tools for analyzing Erlang applications:

# CPU

• erl-on-cpu: Erlang-Land on-CPU Flame Graph

# off-CPU

• erl-off-cpu: Erlang-Land off-CPU Flame Graph

# Go

OpenResty XRay provides the following tools for analyzing Go applications:

# CPU

• go-alloc-fgraph: Go Memory Allocation Flame Graph
• go-excep-fgraph: Go-Land Exception Flame Graph
• go-live-goroutines: dump the stacks of the living goroutines
• go-on-cpu: Go-Land on-CPU Flame Graph

# Latency

• go-net-http-req-latency: Go net/http Request Latency

# Memory

• go-gco-ref: Go-Land GC Object Reference Flame Graph
• go-memory: Go-Land Memory Breakdown

# Miscellaneous

• go-core-dump-analysis: Go Application Core Dump Analysis

# Network I/O

• krakend-req-canceled: Krakend actively canceled requests

# off-CPU

• go-off-cpu: Go-Land off-CPU Flame Graph

# Java

OpenResty XRay provides the following tools for analyzing Java applications:

# CPU

• java-find-method-entry: Find Java method entries
• java-ftrace: Trace Java function call entry events
• java-on-cpu: Java-Land on-CPU Flame Graph
• pre-java-ftrace: Preliminary Analyzer for tracing Java function call entry events

# Disk I/O

• java-read-fgraph: Java-Land Virtual File System Read/Sendfile Flame Graph
• java-read-latency-fgraph: Java-Land Virtual File System Read/Sendfile Latency Flame Graph
• java-read-volume-fgraph: Java-Land Virtual File System Read/Sendfile Volume Flame Graph
• java-write-fgraph: Java-Land Virtual File System Write Flame Graph
• java-write-latency-fgraph: Java-Land Virtual File System Write Latency Flame Graph
• java-write-volume-fgraph: Java-Land Virtual File System Write Volume Flame Graph

# Memory

• java-gco-ref: Java-Land GC Object Reference Flame Graph
• java-newgco: Java GC Object Allocation Flame Graph
• java-newgco-size: Java GC Object Allocation Size Flame Graph

# Network I/O

• java-kafka-req-latency: Java Kafka Request Latency
• java-tomcat-req-latency: Java Tomcat Request Latency
• java-tomcat-top-latency-reqs: IP packets of top latency HTTP requests
• pre-java-kafka-req-latency: Preliminary Analyzer for Java Kafka Request Latency
• pre-java-tomcat-req-latency: Preliminary Analyzer for Java Tomcat Request Latency

# off-CPU

• java-off-cpu: Java-Land off-CPU Flame Graph

# Kong

OpenResty XRay provides the following tools for analyzing Kong applications:

# CPU

• lj-add-timer-lua-fgraph: ngx.timer Add Timer Flamegraph
• lj-c-on-cpu: C-Land CPU Flame Graph for LuaJIT
• lj-dump-traces: Dump LuaJIT Traces
• lj-excep-lua-fgraph: Lua Exception Flame Graph
• lj-func-events: Trace LuaJIT function call entry events
• lj-gc-step-calls: Number of LuaJIT GC function calls per second
• lj-jit-state: Check The LuaJIT JIT State: on/off
• lj-live-coroutines: dump the bts of the living Lua coroutines
• lj-lua-full-bt: Dump Lua Full Backtrace including the variables in the stack
• lj-lua-newcdata: Lua-Land New CData Creation Flame Graph Sampling Tool for LuaJIT Applications.
• lj-lua-newfunc: Lua-Land New Function Creation Flame Graph sampling Tool for LuaJIT Applications.
• lj-lua-newgco: LuaJIT GC Object Allocation Flame Graph
• lj-lua-newstr: LuaJIT String Objects Allocation Flame Graph
• lj-lua-newtab: LuaJIT Table Objects Allocation Flame Graph
• lj-lua-newthread: LuaJIT Thread Objects Allocation Flame Graph
• lj-lua-on-cpu: Lua-Land CPU Flame Graph
• lj-lua-tab-resize: LuaJIT Table resize Flame Graph
• lj-no-pcre-jit-lua-fgraph: Lua Land Flame Graph for function path that PCRE jit is disabled
• lj-slowest-ngx-re: Slowest find/match/split Regular Expression of ngx.re
• lj-slowest-str-gmatch: Slowest Lua string.gmatch
• lj-slowest-str-gsub: Slowest String Global Substitution For Lua’s string.gsub
• lj-slowest-str-match-find: Slowest String Match/Find
• lj-str-find-no-plain-fgraph: Lua string.find Searching Plain String Without Plain Mode Set
• lj-trace-abort-events-fgraph: Lua Land Trace Abort Events Flame Graph
• lj-trace-stats: LuaJIT VM Trace Event Statistics
• lj-vm-states: LuaJIT VM States Distribution
• ngx-access-log-buffer-size: Nginx Access Log Buffer Size Configuration
• ngx-brotli-compression-level: Nginx Brotli Compression Level Statistics
• ngx-brotli-compression-types: Nginx Brotli Compression Types Statistics
• ngx-cpu-hottest-hosts: Showing the top 10 hostnames consuming the most CPU resources
• ngx-cpu-hottest-uris: Showing the top 10 URIs order by request count
• ngx-epoll-event-stats: Statistics of Epoll Events Received
• ngx-epoll-wait-timers: Tracking the Nginx-Land Epoll Wait Timeout Values
• ngx-gzip-compression-level: Nginx GZIP Compression Level Statistics
• ngx-gzip-compression-types: Nginx GZIP Compression Types Statistics
• ngx-lua-code-cache: Nginx Lua Code Cache Requests
• ngx-open-file-cache-conf: Nginx Open File Cache Configuration
• ngx-pcre-compile-lua-fgraph: Openresty New PCRE Compilation Lua Land Flame Graph
• ngx-shdict-hit-rate: Nginx Shared Dict Hit Rate
• ngx-slowest-config-re: Slowest Regular Expression in Nginx Rewrite or Find Config
• ngx-stats: Nginx Statistics from nginx_status
• ngx-timer-delay-distr: Timeout Value Distribution of Nginx Timer
• ngx-zstd-compression-level: Nginx ZStandard Compression Level Statistics
• ngx-zstd-compression-types: Nginx ZStandard Compression Types Statistics
• openresty-compiled-regex-cache: OpenResty Compiled Regex Cache Statistics
• openresty-cosocket-handshake: Cosocket SSL Handshake Flame Graph
• openresty-new-timer-fgraph: Openresty New Timer Flamegraph
• openresty-running-timers-fgraph: Openresty Running Timer Flamegraph

# Disk I/O

• lj-read-fgraph: Lua-Land Virtual File System Read/Sendfile Flame Graph
• lj-read-latency-fgraph: Lua-Land Virtual File System Read/Sendfile Latency Flame Graph
• lj-read-volume-fgraph: Lua-Land Virtual File System Read/Sendfile Volume Flame Graph
• lj-write-fgraph: Lua-Land Virtual File System Write Flame Graph
• lj-write-latency-fgraph: Lua-Land Virtual File System Write Latency Flame Graph
• lj-write-volume-fgraph: Lua-Land Virtual File System Write Volume Flame Graph

# Error

• lj-lua-ngx-print-error: ngx.print and ngx.say error Flame Graph

# Latency

• ngx-err-log-files: Nginx Error Log Files
• ngx-http-access-time: Nginx HTTP Access Phase Latency
• ngx-http-rewrite-time: Nginx HTTP Rewrite Phase Latency
• ngx-http-upstream-time: Nginx HTTP Upstream Latency
• ngx-https-handshake-time: Nginx HTTPs handshake - useless
• ngx-read-req-body-time: Nginx Request Body Reading Time Distribution
• ngx-req-latency: Nginx Request Latency
• ngx-req-latency-advance: Nginx Request Latency with Lua Profiling
• ngx-req-latency-distr: HTTP Request Latency Distribution
• ngx-req-lj-yield-count: Nginx Request LuaJIT Yield Times Flame Graph
• ngx-req-lj-yield-latency: Nginx Request LuaJIT Yield Latency Flame Graph
• ngx-req-lj-yield-time: Nginx Request LuaJIT Yield Latency
• ngx-tls-handshake-latency: Nginx SSL/TLS handshake Latency
• openresty-coro-timeout-packets: Nginx Request Latency
• sema-wait-fgraph: Semaphore wait Calls

# Memory

• lj-alloc-stats: Statistics for LuaJIT’s builtin Allocator (Directly allocated memory blocks larger than 128K will not be counted)
• lj-c-memory-leak-fgraph: Lua-Land Memory Leak Flame Graphs
• lj-dump-cdata-fin: Dump LuaJIT CDATA finalizer
• lj-dump-loaded-mods: Showing the loaded modules of the LuaJIT VM
• lj-err-mem: Luajit Memory Exception
• lj-free-stats: Statistics of free chunks for LuaJIT’s builtin Allocator
• lj-gco-cdata-fin: CData Object Statistics by Finalizer
• lj-gco-cdata-stats: CData Object Statistics by CData Type
• lj-gco-ref: GC Object Reference Flame Graph
• lj-gco-stat: Statistics for LuaJIT GC Objects
• lj-lua-newgco-size: LuaJIT GC Object Allocation Size Flame Graph
• lj-mem-realloc-distr: Lua Memory Realloc Size Distribution
• lj-mem-realloc-fgraph: Lua Memory Realloc Size Flamegraphs
• lj-tab-chain-loops: Find loops in LuaJIT table node chains.
• lj-tab-unreachable-node: Find unreachable nodes in the lua tables.
• ngx-loaded-modules: Nginx Loaded Modules
• ngx-mem-pools: Nginx Memory Pool Statistics
• ngx-shdict-stores: Nginx Shdict Stores Statistics
• ngx-shm-lru: Nginx Share Memory LRU Statistics
• ngx-shm-slabs: Slab Memory Usage in Nginx Shared Memory
• ngx-shm-tree-height: Compute the RBTree Height of Nginx Shared Memory Zones.
• resty-memory: Application-Level Memory Usage Breakdown

# Miscellaneous

• collect-luajit-ffnames: Collect LuaJIT ffnames
• lj-gc-state: LuaJIT GC configuration and runtime state
• lj-lua-new-timer-errors: Nginx Lua Tracks Timer Creation Errors
• ngx-config: Configurations for Nginx
• ngx-config-servers: Dump The Name of The Http Servers
• ngx-dump-timers: Dump nginx timers in the rbtree
• ngx-graceful-exits: Nginx Graceful Exit Event
• ngx-lua-sleep: Nginx Lua Sleep Calls Distribution
• ngx-lua-sleep-fgraph: Nginx Lua Sleep Flame Graph
• ngx-worker-id: Nginx Worker ID
• openresty-core-dump-analysis: OpenResty Core Dump Analysis

# Network I/O

• lj-lua-ngx-exit-fgraph: Lua-Land Nginx Request Status Code Flame Graph
• lj-lua-socket-close-fgraph: Lua-Land close socket Flame Graph
• ngx-downstream-keepalive-stats: HTTP Downstream Request Connection Keepalive Statistics
• ngx-downstream-ssl-session-reused: Nginx Downstream SSL/TLS Session Reused Statistics
• ngx-dump-cur-req: Dump Nginx Request
• ngx-dump-dns: Dump Nginx DNS configuration and domain caches
• ngx-dump-req: Dump Nginx Request
• ngx-keepalive-statistics: Request/Upstream Connection Keep-alive Statistics
• ngx-proxy-cache-status: Proxy Cache Status Distribution
• ngx-read-req-body-packets: Capture the packets of the HTTP request that the read body time is greater than the threshold
• ngx-req-count: Count Nginx Subrequests and Main Requests
• ngx-req-counts-by-hosts: Showing the top 10 hostnames order by request counts
• ngx-req-distribution: Nginx Traffic Distribution By Calls of Finalize Request
• ngx-req-err-packets: IP Packets of Specified HTTP Errors
• ngx-req-size-by-hosts: Showing the top 10 hostnames order by request size
• ngx-req-status-code: Request Status Code Per Second
• ngx-rps: Nginx RPS Statistics
• ngx-ssl-server-conf: Sample the Server Configurations by Requests
• ngx-ssl-session-ticket-key: Nginx SSL Session Ticket Key Configuration
• ngx-status-code-fgraph: Nginx Request Status Code Flame Graph
• ngx-tls-handshake-info: SSL/TLS Session handshake information Statistics
• ngx-tls-session-resumption: Downstream SSL/TLS Session Resumption Statistics
• ngx-top-latency-reqs: IP packets of top latency HTTP requests
• ngx-upstream-err-packets: Capture the upstream packets that have specific error code
• ngx-upstream-no-keepalive: capture the packets that have no HTTP upstream ‘keepalive’ configuration
• ngx-upstream-ssl-session-reused: Upstream SSL/TLS Session Reused Statistics
• openresty-cosocket-connect: OpenResty Cosocket Connect Operation Flame Graph
• openresty-cosocket-err-fgraph: OpenResty Cosocket Error Flame Graph
• openresty-cosocket-err-packets: IP Packets of OpenResty Cosocket Which has Errors
• openresty-cosocket-no-keepalive: OpenResty Cosocket Without Keepalive Flame Graph
• openresty-cosocket-pool: OpenResty Cosocket Pool Statistics
• openresty-cosocket-timeout: OpenResty Cosocket Timeout Distribution
• openresty-lua-phase-cpu-time: OpenResty Lua Phase CPU time statistics
• openresty-lua-phase-wall-time: OpenResty Lua Phase wall time statistics
• socket-close-fgraph: close socket Flame Graph

# off-CPU

• lj-c-off-cpu: C-Land off-CPU Flame Graph for LuaJIT
• lj-lua-off-cpu: Lua-Land off-CPU Flame Graph
• ngx-err-log-lvl-distr: Nginx Error Log Level Distribution

# Lua

OpenResty XRay provides the following tools for analyzing Lua applications:

# CPU

• lua-excep-fgraph: Lua-Land Exception Flame Graph for standard Lua
• lua-on-cpu: Lua-Land CPU Flame Graph for standard Lua

# Memory

• lua-gco-ref: GC Object Reference Flame Graph for Stand Lua

# off-CPU

• lua-off-cpu: Lua-Land off-CPU Flame Graph for standard Lua

# Nginx

OpenResty XRay provides the following tools for analyzing Nginx applications:

# CPU

• ngx-access-log-buffer-size: Nginx Access Log Buffer Size Configuration
• ngx-brotli-compression-level: Nginx Brotli Compression Level Statistics
• ngx-brotli-compression-types: Nginx Brotli Compression Types Statistics
• ngx-epoll-event-stats: Statistics of Epoll Events Received
• ngx-epoll-wait-timers: Tracking the Nginx-Land Epoll Wait Timeout Values
• ngx-gzip-compression-level: Nginx GZIP Compression Level Statistics
• ngx-gzip-compression-types: Nginx GZIP Compression Types Statistics
• ngx-open-file-cache-conf: Nginx Open File Cache Configuration
• ngx-slowest-config-re: Slowest Regular Expression in Nginx Rewrite or Find Config
• ngx-zstd-compression-level: Nginx ZStandard Compression Level Statistics
• ngx-zstd-compression-types: Nginx ZStandard Compression Types Statistics

# Latency

• ngx-err-log-files: Nginx Error Log Files
• ngx-https-handshake-time: Nginx HTTPs handshake - useless
• ngx-req-latency: Nginx Request Latency
• ngx-req-latency-advance: Nginx Request Latency with Lua Profiling
• ngx-req-latency-distr: HTTP Request Latency Distribution

# Memory

• ngx-mem-pools: Nginx Memory Pool Statistics
• ngx-shm-slabs: Slab Memory Usage in Nginx Shared Memory

# Miscellaneous

• ngx-config: Configurations for Nginx
• ngx-dump-timers: Dump nginx timers in the rbtree
• ngx-graceful-exits: Nginx Graceful Exit Event
• ngx-worker-id: Nginx Worker ID

# Network I/O

• ngx-dump-all-reqs: Dump all Nginx HTTP Requests
• ngx-read-req-body-packets: Capture the packets of the HTTP request that the read body time is greater than the threshold
• ngx-req-distribution: Nginx Traffic Distribution By Calls of Finalize Request
• ngx-req-err-packets: IP Packets of Specified HTTP Errors
• ngx-ssl-session-ticket-key: Nginx SSL Session Ticket Key Configuration
• ngx-status-code-fgraph: Nginx Request Status Code Flame Graph
• ngx-top-latency-reqs: IP packets of top latency HTTP requests
• ngx-upstream-err-packets: Capture the upstream packets that have specific error code
• ngx-upstream-no-keepalive: capture the packets that have no HTTP upstream ‘keepalive’ configuration

# off-CPU

• ngx-err-log-lvl-distr: Nginx Error Log Level Distribution

# Node.js

OpenResty XRay provides the following tools for analyzing Node.js applications:

# CPU

• nodejs-ftrace: Trace Node.js function call entry events
• nodejs-on-cpu: Node JS-Land on-CPU Flame Graph
• pre-nodejs-ftrace: Preliminary Analyzer for tracing Node.js function call entry events

# off-CPU

• nodejs-off-cpu: Node JS-Land off-CPU Flame Graph

# OpenResty

OpenResty XRay provides the following tools for analyzing OpenResty applications:

# CPU

• lj-add-timer-lua-fgraph: ngx.timer Add Timer Flamegraph
• lj-c-on-cpu: C-Land CPU Flame Graph for LuaJIT
• lj-coro-c-on-cpu: C-Land CPU Flame Graph for LuaJIT and Coro
• lj-dump-traces: Dump LuaJIT Traces
• lj-excep-lua-fgraph: Lua Exception Flame Graph
• lj-func-events: Trace LuaJIT function call entry events
• lj-gc-step-calls: Number of LuaJIT GC function calls per second
• lj-jit-state: Check The LuaJIT JIT State: on/off
• lj-live-coroutines: dump the bts of the living Lua coroutines
• lj-lua-full-bt: Dump Lua Full Backtrace including the variables in the stack
• lj-lua-newcdata: Lua-Land New CData Creation Flame Graph Sampling Tool for LuaJIT Applications.
• lj-lua-newfunc: Lua-Land New Function Creation Flame Graph sampling Tool for LuaJIT Applications.
• lj-lua-newgco: LuaJIT GC Object Allocation Flame Graph
• lj-lua-newstr: LuaJIT String Objects Allocation Flame Graph
• lj-lua-newtab: LuaJIT Table Objects Allocation Flame Graph
• lj-lua-newthread: LuaJIT Thread Objects Allocation Flame Graph
• lj-lua-on-cpu: Lua-Land CPU Flame Graph
• lj-lua-tab-resize: LuaJIT Table resize Flame Graph
• lj-no-pcre-jit-lua-fgraph: Lua Land Flame Graph for function path that PCRE jit is disabled
• lj-slowest-ngx-re: Slowest find/match/split Regular Expression of ngx.re
• lj-slowest-str-gmatch: Slowest Lua string.gmatch
• lj-slowest-str-gsub: Slowest String Global Substitution For Lua’s string.gsub
• lj-slowest-str-match-find: Slowest String Match/Find
• lj-str-find-no-plain-fgraph: Lua string.find Searching Plain String Without Plain Mode Set
• lj-trace-abort-events-fgraph: Lua Land Trace Abort Events Flame Graph
• lj-trace-stats: LuaJIT VM Trace Event Statistics
• lj-vm-states: LuaJIT VM States Distribution
• ngx-access-log-buffer-size: Nginx Access Log Buffer Size Configuration
• ngx-brotli-compression-level: Nginx Brotli Compression Level Statistics
• ngx-brotli-compression-types: Nginx Brotli Compression Types Statistics
• ngx-cpu-hottest-hosts: Showing the top 10 hostnames consuming the most CPU resources
• ngx-cpu-hottest-uris: Showing the top 10 URIs order by request count
• ngx-epoll-event-stats: Statistics of Epoll Events Received
• ngx-epoll-wait-timers: Tracking the Nginx-Land Epoll Wait Timeout Values
• ngx-gzip-compression-level: Nginx GZIP Compression Level Statistics
• ngx-gzip-compression-types: Nginx GZIP Compression Types Statistics
• ngx-lua-code-cache: Nginx Lua Code Cache Requests
• ngx-open-file-cache-conf: Nginx Open File Cache Configuration
• ngx-pcre-compile-lua-fgraph: Openresty New PCRE Compilation Lua Land Flame Graph
• ngx-shdict-hit-rate: Nginx Shared Dict Hit Rate
• ngx-slowest-config-re: Slowest Regular Expression in Nginx Rewrite or Find Config
• ngx-stats: Nginx Statistics from nginx_status
• ngx-timer-delay-distr: Timeout Value Distribution of Nginx Timer
• ngx-zstd-compression-level: Nginx ZStandard Compression Level Statistics
• ngx-zstd-compression-types: Nginx ZStandard Compression Types Statistics
• openresty-compiled-regex-cache: OpenResty Compiled Regex Cache Statistics
• openresty-cosocket-handshake: Cosocket SSL Handshake Flame Graph
• openresty-new-timer-fgraph: Openresty New Timer Flamegraph
• openresty-running-timers-fgraph: Openresty Running Timer Flamegraph

# Disk I/O

• lj-read-fgraph: Lua-Land Virtual File System Read/Sendfile Flame Graph
• lj-read-latency-fgraph: Lua-Land Virtual File System Read/Sendfile Latency Flame Graph
• lj-read-volume-fgraph: Lua-Land Virtual File System Read/Sendfile Volume Flame Graph
• lj-write-fgraph: Lua-Land Virtual File System Write Flame Graph
• lj-write-latency-fgraph: Lua-Land Virtual File System Write Latency Flame Graph
• lj-write-volume-fgraph: Lua-Land Virtual File System Write Volume Flame Graph

# Error

• lj-lua-ngx-print-error: ngx.print and ngx.say error Flame Graph

# Latency

• ngx-err-log-files: Nginx Error Log Files
• ngx-http-access-time: Nginx HTTP Access Phase Latency
• ngx-http-rewrite-time: Nginx HTTP Rewrite Phase Latency
• ngx-http-upstream-time: Nginx HTTP Upstream Latency
• ngx-https-handshake-time: Nginx HTTPs handshake - useless
• ngx-read-req-body-time: Nginx Request Body Reading Time Distribution
• ngx-req-latency: Nginx Request Latency
• ngx-req-latency-advance: Nginx Request Latency with Lua Profiling
• ngx-req-latency-distr: HTTP Request Latency Distribution
• ngx-req-lj-yield-count: Nginx Request LuaJIT Yield Times Flame Graph
• ngx-req-lj-yield-latency: Nginx Request LuaJIT Yield Latency Flame Graph
• ngx-req-lj-yield-time: Nginx Request LuaJIT Yield Latency
• ngx-tls-handshake-latency: Nginx SSL/TLS handshake Latency
• openresty-coro-timeout-packets: Nginx Request Latency
• sema-wait-fgraph: Semaphore wait Calls

# Memory

• lj-alloc-stats: Statistics for LuaJIT’s builtin Allocator (Directly allocated memory blocks larger than 128K will not be counted)
• lj-c-memory-leak-fgraph: Lua-Land Memory Leak Flame Graphs
• lj-dump-cdata-fin: Dump LuaJIT CDATA finalizer
• lj-dump-loaded-mods: Showing the loaded modules of the LuaJIT VM
• lj-err-mem: Luajit Memory Exception
• lj-free-stats: Statistics of free chunks for LuaJIT’s builtin Allocator
• lj-gco-cdata-fin: CData Object Statistics by Finalizer
• lj-gco-cdata-stats: CData Object Statistics by CData Type
• lj-gco-ref: GC Object Reference Flame Graph
• lj-gco-stat: Statistics for LuaJIT GC Objects
• lj-lua-newgco-size: LuaJIT GC Object Allocation Size Flame Graph
• lj-mem-realloc-distr: Lua Memory Realloc Size Distribution
• lj-mem-realloc-fgraph: Lua Memory Realloc Size Flamegraphs
• lj-tab-chain-loops: Find loops in LuaJIT table node chains.
• lj-tab-unreachable-node: Find unreachable nodes in the lua tables.
• ngx-loaded-modules: Nginx Loaded Modules
• ngx-mem-pools: Nginx Memory Pool Statistics
• ngx-shdict-stores: Nginx Shdict Stores Statistics
• ngx-shm-lru: Nginx Share Memory LRU Statistics
• ngx-shm-slabs: Slab Memory Usage in Nginx Shared Memory
• ngx-shm-tree-height: Compute the RBTree Height of Nginx Shared Memory Zones.
• resty-memory: Application-Level Memory Usage Breakdown

# Miscellaneous

• collect-luajit-ffnames: Collect LuaJIT ffnames
• lj-gc-state: LuaJIT GC configuration and runtime state
• lj-lua-new-timer-errors: Nginx Lua Tracks Timer Creation Errors
• ngx-config: Configurations for Nginx
• ngx-config-servers: Dump The Name of The Http Servers
• ngx-dump-timers: Dump nginx timers in the rbtree
• ngx-graceful-exits: Nginx Graceful Exit Event
• ngx-lua-sleep: Nginx Lua Sleep Calls Distribution
• ngx-lua-sleep-fgraph: Nginx Lua Sleep Flame Graph
• ngx-worker-id: Nginx Worker ID
• openresty-core-dump-analysis: OpenResty Core Dump Analysis

# Network I/O

• lj-lua-ngx-exit-fgraph: Lua-Land Nginx Request Status Code Flame Graph
• lj-lua-socket-close-fgraph: Lua-Land close socket Flame Graph
• ngx-downstream-keepalive-stats: HTTP Downstream Request Connection Keepalive Statistics
• ngx-downstream-ssl-session-reused: Nginx Downstream SSL/TLS Session Reused Statistics
• ngx-dump-all-reqs: Dump all Nginx HTTP Requests
• ngx-dump-cur-req: Dump Nginx Request
• ngx-dump-dns: Dump Nginx DNS configuration and domain caches
• ngx-dump-req: Dump Nginx Request
• ngx-keepalive-statistics: Request/Upstream Connection Keep-alive Statistics
• ngx-proxy-cache-status: Proxy Cache Status Distribution
• ngx-read-req-body-packets: Capture the packets of the HTTP request that the read body time is greater than the threshold
• ngx-req-count: Count Nginx Subrequests and Main Requests
• ngx-req-counts-by-hosts: Showing the top 10 hostnames order by request counts
• ngx-req-distribution: Nginx Traffic Distribution By Calls of Finalize Request
• ngx-req-err-packets: IP Packets of Specified HTTP Errors
• ngx-req-size-by-hosts: Showing the top 10 hostnames order by request size
• ngx-req-status-code: Request Status Code Per Second
• ngx-rps: Nginx RPS Statistics
• ngx-ssl-server-conf: Sample the Server Configurations by Requests
• ngx-ssl-session-ticket-key: Nginx SSL Session Ticket Key Configuration
• ngx-status-code-fgraph: Nginx Request Status Code Flame Graph
• ngx-tls-handshake-info: SSL/TLS Session handshake information Statistics
• ngx-tls-session-resumption: Downstream SSL/TLS Session Resumption Statistics
• ngx-top-latency-reqs: IP packets of top latency HTTP requests
• ngx-upstream-err-packets: Capture the upstream packets that have specific error code
• ngx-upstream-no-keepalive: capture the packets that have no HTTP upstream ‘keepalive’ configuration
• ngx-upstream-ssl-session-reused: Upstream SSL/TLS Session Reused Statistics
• openresty-cosocket-connect: OpenResty Cosocket Connect Operation Flame Graph
• openresty-cosocket-err-fgraph: OpenResty Cosocket Error Flame Graph
• openresty-cosocket-err-packets: IP Packets of OpenResty Cosocket Which has Errors
• openresty-cosocket-no-keepalive: OpenResty Cosocket Without Keepalive Flame Graph
• openresty-cosocket-pool: OpenResty Cosocket Pool Statistics
• openresty-cosocket-timeout: OpenResty Cosocket Timeout Distribution
• openresty-lua-phase-cpu-time: OpenResty Lua Phase CPU time statistics
• openresty-lua-phase-wall-time: OpenResty Lua Phase wall time statistics
• socket-close-fgraph: close socket Flame Graph

# off-CPU

• lj-c-off-cpu: C-Land off-CPU Flame Graph for LuaJIT
• lj-lua-off-cpu: Lua-Land off-CPU Flame Graph
• ngx-err-log-lvl-distr: Nginx Error Log Level Distribution

# Perl

OpenResty XRay provides the following tools for analyzing Perl applications:

# CPU

• perl-excep-fgraph: Perl-Land Exception Flame Graph
• perl-func-events: Trace perl function call entry events
• perl-on-cpu: Perl-Land on-CPU Flame Graph

# Disk I/O

• perl-read-fgraph: Perl-Land Virtual File System Read/Sendfile Flame Graph
• perl-read-latency-fgraph: Perl-Land Virtual File System Read/Sendfile Latency Flame Graph
• perl-read-volume-fgraph: Perl-Land Virtual File System Read/Sendfile Volume Flame Graph
• perl-write-fgraph: Perl-Land Virtual File System Write Flame Graph
• perl-write-latency-fgraph: Perl-Land Virtual File System Write Latency Flame Graph
• perl-write-volume-fgraph: Perl-Land Virtual File System Write Volume Flame Graph

# Memory

• perl-gco-ref: Perl GC Object Reference Flame Graph

# off-CPU

• perl-off-cpu: Perl-Land off-CPU Flame Graph

# PHP

OpenResty XRay provides the following tools for analyzing PHP applications:

# CPU

• php-excep-fgraph: PHP-Land Exception Flame Graph
• php-func-events: Trace PHP function call entry events
• php-on-cpu: PHP-Land CPU Flame Graph

# Disk I/O

• php-read-fgraph: PHP-Land Virtual File System Read/Sendfile Flame Graph
• php-read-latency-fgraph: PHP-Land Virtual File System Read/Sendfile Latency Flame Graph
• php-read-volume-fgraph: PHP-Land Virtual File System Read/Sendfile Volume Flame Graph
• php-write-fgraph: PHP-Land Virtual File System Write Flame Graph
• php-write-latency-fgraph: PHP-Land Virtual File System Write Latency Flame Graph
• php-write-volume-fgraph: PHP-Land Virtual File System Write Volume Flame Graph

# Memory

• php-bin-distr: Distributions of Memory Chunk Sizes in PHP Zend Allocator
• php-gco-ref: PHP GC Object Reference Flame Graph
• php-gco-stat: Statistics for PHP GC Objects

# off-CPU

• php-off-cpu: PHP-Land off-CPU Flame Graph

# PostgreSQL

OpenResty XRay provides the following tools for analyzing PostgreSQL applications:

# Memory

• pg-mem-pool: PostgreSQL Memory Pool Size

# Python

OpenResty XRay provides the following tools for analyzing Python applications:

# CPU

• py-excep-fgraph: Python-Land Exception Flame Graph
• py-func-events: Trace python function call entry events
• py-on-cpu: Python CPU Flame Graph
• pyftrace: Trace python function call entry/exit events

# Disk I/O

• py-read-fgraph: Python-Land Virtual File System Read/Sendfile Flame Graph
• py-read-latency-fgraph: Python-Land Virtual File System Read/Sendfile Latency Flame Graph
• py-read-volume-fgraph: Python-Land Virtual File System Read/Sendfile Volume Flame Graph
• py-write-fgraph: Python-Land Virtual File System Write Flame Graph
• py-write-latency-fgraph: Python-Land Virtual File System Write Latency Flame Graph
• py-write-volume-fgraph: Python-Land Virtual File System Write Volume Flame Graph

# Memory

• pre-py-gco-ref: Preliminary Analyzer for Python GC Objects Reference Flame Graph
• py-gco-ref: Python GC Object Reference Flame Graph
• py-gco-stat: Statistics for Python GC Objects

# off-CPU

• py-off-cpu: Python3 off-CPU Flame Graph

# Redis

OpenResty XRay provides the following tools for analyzing Redis applications:

# CPU

• lua-excep-fgraph: Lua-Land Exception Flame Graph for standard Lua
• lua-on-cpu: Lua-Land CPU Flame Graph for standard Lua

# Memory

• lua-gco-ref: GC Object Reference Flame Graph for Stand Lua

# off-CPU

• lua-off-cpu: Lua-Land off-CPU Flame Graph for standard Lua

# Ruby

OpenResty XRay provides the following tools for analyzing Ruby applications:

# CPU

• ruby-on-cpu: Ruby CPU Flame Graph

# off-CPU

• ruby-off-cpu: Ruby off-CPU Flame Graph

# Rust

OpenResty XRay provides the following tools for analyzing Rust applications:

# CPU

• rust-excep-fgraph: Rust-Land Exception Flame Graph
• rust-on-cpu: Rust-Land on-CPU Flame Graph

# off-CPU

• rust-off-cpu: Rust-Land off-CPU Flame Graph

# Author

The OpenResty Inc. Team.

# Copyright

Copyright (C) by OpenResty Inc. All rights reserved.