4 分鐘閱讀 | 最後修改: 0001-01-01, 0:00:00 am

lua-resty-http-fast

Table of Contents

Name

lua-resty-http-fast - Lua HTTP client for OpenResty

Status

This library is considered production ready.

Description

This Lua library is a HTTP client driver for the ngx_lua nginx module:

https://github.com/openresty/lua-nginx-module/#readme

This Lua library takes advantage of ngx_lua’s cosocket API, which ensures 100% nonblocking behavior.

Note that at least ngx_lua 0.10.26 or OpenResty 1.21.4.3 is required.

Back to TOC

Synopsis

# The load_module directive must be on top of nginx.conf
load_module "/usr/local/openresty-coro-nginx-module/lib/ngx_http_coro_module.so";
load_module "/usr/local/openresty-coro-nginx-module/lib/ngx_http_coro_libcurl_module.so";

...

http {
    # you do not need the following line if you are using
    # the OpenResty bundle:
    lua_package_path "/usr/local/openresty/site/lualib/?.ljbc;;";

    # specify the location of the libcurl
    coro_preload /usr/local/openresty/openssl111/lib/libcrypto.so;
    coro_preload /usr/local/openresty/openssl111/lib/libssl.so;
    coro_preload /usr/local/openresty/libcurl/lib/libcurl.so;

    coro_stack_size  65536;

    server {
        location /a {
            # need to specify the resolver to resolve the hostname
            resolver 8.8.8.8;

            content_by_lua_block {
                local http = require "resty.http.fast"
                local httpc = http.new()
                httpc:connect{
                    scheme = "http",
                    host = "127.0.0.1",
                    port = ngx.var.server_port
                }

                local res, err = httpc:request{
                    path = "/b"
                }

                ngx.status = res.status
                ngx.print(res:read_body())

                httpc:close()
            }
        }

        location = /b {
            echo "OK";
        }
    }
}

Back to TOC

Features

  • HTTP 1.0 and 1.1
  • SSL
  • Alternative simple interface for single-shot requests without a manual connection step
  • Connection keepalives
  • Request pipelining
  • mTLS (requires ngx_lua_http_module >= v0.10.23)

TODO

  • Streaming interface to the response body, for predictable memory usage
  • Chunked transfer encodings
  • HTTP proxy connections
  • Trailers

Back to TOC

API

new

syntax: httpc, err = http.new()

Creates the HTTP connection object. In case of failures, returns nil and a string describing the error.

Back to TOC

connect

syntax: ok, err, ssl_session = httpc:connect(options)

Attempts to connect to the web server while incorporating the following activities:

  • TCP connection
  • SSL handshake

In doing so it will create a distinct connection pool name that is safe to use with SSL and / or proxy based connections.

The options table has the following fields:

  • scheme: scheme to use, or nil for unix domain socket
  • host: target host, or path to a unix domain socket
  • port: port on target host, will default to 80 or 443 based on the scheme
  • pool: custom connection pool name. Except that the default will become a pool name constructed using the SSL / proxy properties, which is important for safe connection reuse. When in doubt, leave it blank!
  • pool_size: maximum number of idle connections in the pool
  • backlog: maximum number of waiting requests when the connection pool is full
  • ssl_reused_session: a former SSL session object which can be reused
  • ssl_verify: whether to perform SSL verification, defaults to true
  • ssl_server_ca_pem: PEM-formatted CA certificate chain for SSL server certificate verification
  • ssl_server_name: the server name for the new TLS extension Server Name Indication (SNI)
  • ssl_client_cert_pem: client certificate data in PEM format for SSL/TLS authentication
  • ssl_client_priv_key_pem: client private key data in PEM format for SSL/TLS authentication

Back to TOC

set_timeout

syntax: httpc:set_timeout(time)

Sets the socket timeout (in ms) for subsequent operations. See set_timeouts below for a more declarative approach.

Back to TOC

set_timeouts

syntax: httpc:set_timeouts(connect_timeout, send_timeout, read_timeout)

Sets the connect timeout threshold, send timeout threshold, and read timeout threshold, respectively, in milliseconds, for subsequent socket operations (connect, send, receive, and iterators returned from receiveuntil).

Back to TOC

set_keepalive

syntax: ok, err = httpc:set_keepalive(max_idle_timeout, pool_size)

Either places the current connection into the pool for future reuse, or closes the connection. Calling this instead of close is “safe” in that it will conditionally close depending on the type of request. Specifically, a 1.0 request without Connection: Keep-Alive will be closed, as will a 1.1 request with Connection: Close.

In case of success, returns 1. In case of errors, returns nil, err. In the case where the connection is conditionally closed as described above, returns 2 and the error string connection must be closed, so as to distinguish from unexpected errors.

See OpenResty docs for parameter documentation.

Back to TOC

get_reused_times

syntax: times, err = httpc:get_reused_times()

See OpenResty docs.

Back to TOC

close

syntax: ok, err = httpc:close()

See OpenResty docs.

Back to TOC

request

syntax: res, err = httpc:request(params)

Sends an HTTP request over an already established connection. Returns a res table or nil and an error message.

The params table expects the following fields:

  • version: The HTTP version number. Defaults to 1.1.
  • method: The HTTP method string. Defaults to GET.
  • path: The path string. Defaults to /.
  • query: The query string, presented as either a literal string or Lua table..
  • headers: A table of request headers.
  • body: The request body as a string, a table of strings, or an iterator function yielding strings until nil when exhausted. Note that you must specify a Content-Length for the request body, or specify Transfer-Encoding: chunked and have your function implement the encoding. See also: get_client_body_reader).

When the request is successful, res will contain the following fields:

  • status: The status code.
  • reason: The status reason phrase.
  • headers: A table of headers. Multiple headers with the same field name will be presented as a table of values.
  • has_body: A boolean flag indicating if there is a body to be read.
  • body_reader: An iterator function for reading the body in a streaming fashion.
  • read_body: A method to read the entire body into a string.
  • read_trailers: A method to merge any trailers underneath the headers, after reading the body.

If the response has a body, then before the same connection can be used for another request, you must read the body using read_body or body_reader.

Back to TOC

request_uri

syntax: res, err = httpc:request_uri(uri, params)

The single-shot interface (see usage). Since this method performs an entire end-to-end request, options specified in the params can include anything found in both connect and request documented above. Note also that fields path, and query, in params will override relevant components of the uri if specified (scheme, host, and port will always be taken from the uri).

There are 3 additional parameters for controlling keepalives:

  • keepalive: Set to false to disable keepalives and immediately close the connection. Defaults to true.
  • keepalive_timeout: The maximal idle timeout (ms). Defaults to lua_socket_keepalive_timeout.
  • keepalive_pool: The maximum number of connections in the pool. Defaults to lua_socket_pool_size.

If the request is successful, res will contain the following fields:

  • status: The status code.
  • headers: A table of headers.
  • body: The entire response body as a string.

Back to TOC

request_pipeline

syntax: responses, err = httpc:request_pipeline(params)

This method works as per the request method above, but params is instead a nested table of parameter tables. Each request is sent in order, and responses is returned as a table of response handles. For example:

local responses = httpc:request_pipeline({
    { path = "/b" },
    { path = "/c" },
    { path = "/d" },
})

for _, r in ipairs(responses) do
    if not r.status then
        ngx.log(ngx.ERR, "socket read error")
        break
    end

    ngx.say(r.status)
    ngx.say(r:read_body())
end

Due to the nature of pipelining, no responses are actually read until you attempt to use the response fields (status / headers etc). And since the responses are read off in order, you must read the entire body (and any trailers if you have them), before attempting to read the next response.

Note this doesn’t preclude the use of the streaming response body reader. Responses can still be streamed, so long as the entire body is streamed before attempting to access the next response.

Be sure to test at least one field (such as status) before trying to use the others, in case a socket read error has occurred.

Back to TOC

res.body_reader

The body_reader iterator can be used to stream the response body in chunk sizes of your choosing, as follows:

local reader = res.body_reader
local buffer_size = 8192

repeat
    local buffer, err = reader(buffer_size)
    if err then
        ngx.log(ngx.ERR, err)
        break
    end

    if buffer then
        -- process
    end
until not buffer

If the reader is called with no arguments, the behaviour depends on the type of connection. If the response is encoded as chunked, then the iterator will return the chunks as they arrive. If not, it will simply return the entire body.

Note that the size provided is actually a maximum size. So in the chunked transfer case, you may get buffers smaller than the size you ask, as a remainder of the actual encoded chunks.

Back to TOC

res:read_body

syntax: body, err = res:read_body()

Reads the entire body into a local string.

Back to TOC

parse_uri

syntax: local scheme, host, port, path, query? = unpack(httpc:parse_uri(uri, query_in_path?))

This is a convenience function allowing one to more easily use the generic interface, when the input data is a URI.

As of version 0.10, the optional query_in_path parameter was added, which specifies whether the querystring is to be included in the path return value, or separately as its own return value. This defaults to true in order to maintain backwards compatibility. When set to false, path will only include the path, and query will contain the URI args, not including the ? delimiter.

Back to TOC

get_client_body_reader

syntax: reader, err = httpc:get_client_body_reader(chunksize?, sock?)

Returns an iterator function which can be used to read the downstream client request body in a streaming fashion. You may also specify an optional default chunksize (default is 65536), or an already established socket in place of the client request.

Example:

local req_reader = httpc:get_client_body_reader()
local buffer_size = 8192

repeat
    local buffer, err = req_reader(buffer_size)
    if err then
        ngx.log(ngx.ERR, err)
        break
    end

    if buffer then
        -- process
    end
until not buffer

This iterator can also be used as the value for the body field in request params, allowing one to stream the request body into a proxied upstream request.

local client_body_reader, err = httpc:get_client_body_reader()

local res, err = httpc:request({
    path = "/helloworld",
    body = client_body_reader,
})

Back to TOC

Installation

Before using this library, you need to install lua-resty-http-fast, openresty-libcurl and openresty-coro-libcurl-nginx-module packages from our OpenResty XRay private repository. And the nginx.conf file should be configured as follows, load the corresponding nginx dynamic modules and libcurl modules.

# The load_module directive must be on top of nginx.conf
load_module "/usr/local/openresty-coro-nginx-module/lib/ngx_http_coro_module.so";
load_module "/usr/local/openresty-coro-nginx-module/lib/ngx_http_coro_libcurl_module.so";

...

http {
    # specify the location of the libcurl
    coro_preload /usr/local/openresty/openssl111/lib/libcrypto.so;
    coro_preload /usr/local/openresty/openssl111/lib/libssl.so;
    coro_preload /usr/local/openresty/libcurl/lib/libcurl.so;

    coro_stack_size  65536;

    # you do not need the following line if you are using
    # the OpenResty bundle:
    lua_package_path   "/usr/local/openresty/site/lualib/?.ljbc;;";
    ...
}

Ensure that the system account running your Nginx ‘‘worker’’ proceses have enough permission to read the .lua file.

Back to TOC

Copyright & Licenses

Copyright (C) 2024 OpenResty Inc. All rights reserved.

This software is proprietary and must not be redistributed or shared at all.

Back to TOC