• 设为首页
  • 点击收藏
  • 手机版
    手机扫一扫访问
    迪恩网络手机版
  • 关注官方公众号
    微信扫一扫关注
    公众号

p0pr0ck5/lua-resty-waf: High-performance WAF built on the OpenResty stack

原作者: [db:作者] 来自: 网络 收藏 邀请

开源软件名称(OpenSource Name):

p0pr0ck5/lua-resty-waf

开源软件地址(OpenSource Url):

https://github.com/p0pr0ck5/lua-resty-waf

开源编程语言(OpenSource Language):

Perl 73.8%

开源软件介绍(OpenSource Introduction):

Name

lua-resty-waf - High-performance WAF built on the OpenResty stack

Table of Contents

Status

Build Status Codewake CII Best Practices

NOTE: lua-resty-waf is essentially abandoned. This project had a use in a time when ModSecurity for Nginx was not a viable option; this is no longer the case. There was an attempt to revitalize the project in 2020, but I do not have the resources to complete this; this work is partially complete in the redux branch.

Description

lua-resty-waf is a reverse proxy WAF built using the OpenResty stack. It uses the Nginx Lua API to analyze HTTP request information and process against a flexible rule structure. lua-resty-waf is distributed with a ruleset that mimics the ModSecurity CRS, as well as a few custom rules built during initial development and testing, and a small virtual patchset for emerging threats. Additionally, lua-resty-waf is distributed with tooling to automatically translate existing ModSecurity rules, allowing users to extend lua-resty-waf implementation without the need to learn a new rule syntax.

lua-resty-waf was initially developed by Robert Paprocki for his Master's thesis at Western Governor's University.

Requirements

lua-resty-waf requires several third-party resty lua modules, though these are all packaged with lua-resty-waf, and thus do not need to be installed separately. It is recommended to install lua-resty-waf on a system running the OpenResty software bundle; lua-resty-waf has not been tested on platforms built using separate Nginx source and Nginx Lua module packages.

For optimal regex compilation performance, it is recommended to build Nginx/OpenResty with a version of PCRE that supports JIT compilation. If your OS does not provide this, you can build JIT-capable PCRE directly into your Nginx/OpenResty build. To do this, reference the path to the PCRE source in the --with-pcre configure flag. For example:

# ./configure --with-pcre=/path/to/pcre/source --with-pcre-jit

You can download the PCRE source from the PCRE website. See also this blog post for a step-by-step walkthrough on building OpenResty with a JIT-enabled PCRE library.

Performance

lua-resty-waf was designed with efficiency and scalability in mind. It leverages Nginx's asynchronous processing model and an efficient design to process each transaction as quickly as possible. Load testing has show that deployments implementing all provided rulesets, which are designed to mimic the logic behind the ModSecurity CRS, process transactions in roughly 300-500 microseconds per request; this equals the performance advertised by Cloudflare's WAF. Tests were run on a reasonable hardware stack (E3-1230 CPU, 32 GB RAM, 2 x 840 EVO in RAID 0), maxing at roughly 15,000 requests per second. See this blog post for more information.

lua-resty-waf workload is almost exclusively CPU bound. Memory footprint in the Lua VM (excluding persistent storage backed by lua-shared-dict) is roughly 2MB.

Installation

A simple Makefile is provided:

# make && sudo make install

Alternatively, install via Luarocks:

# luarocks install lua-resty-waf

lua-resty-waf makes use of the OPM package manager, available in modern OpenResty distributions. The client OPM tools requires that the resty command line tool is available in your system's PATH environmental variable.

Note that by default lua-resty-waf runs in SIMULATE mode, to prevent immediately affecting an application; users who wish to enable rule actions must explicitly set the operational mode to ACTIVE.

Synopsis

http {
    init_by_lua_block {
        -- use resty.core for performance improvement, see the status note above
        require "resty.core"

        -- require the base module
        local lua_resty_waf = require "resty.waf"

        -- perform some preloading and optimization
        lua_resty_waf.init()
    }

    server {
        location / {
            access_by_lua_block {
                local lua_resty_waf = require "resty.waf"

                local waf = lua_resty_waf:new()

                -- define options that will be inherited across all scopes
                waf:set_option("debug", true)
                waf:set_option("mode", "ACTIVE")

                -- this may be desirable for low-traffic or testing sites
                -- by default, event logs are not written until the buffer is full
                -- for testing, flush the log buffer every 5 seconds
                --
                -- this is only necessary when configuring a remote TCP/UDP
                -- socket server for event logs. otherwise, this is ignored
                waf:set_option("event_log_periodic_flush", 5)

                -- run the firewall
                waf:exec()
            }

            header_filter_by_lua_block {
                local lua_resty_waf = require "resty.waf"

                -- note that options set in previous handlers (in the same scope)
                -- do not need to be set again
                local waf = lua_resty_waf:new()

                waf:exec()
            }

            body_filter_by_lua_block {
                local lua_resty_waf = require "resty.waf"

                local waf = lua_resty_waf:new()

                waf:exec()
            }

            log_by_lua_block {
                local lua_resty_waf = require "resty.waf"

                local waf = lua_resty_waf:new()

                waf:exec()
            }
        }
    }
}

Public Functions

lua-resty-waf.load_secrules()

Translate and initialize a ModSecurity SecRules file from disk. Note that this still requires the ruleset to be added via add_ruleset (the basename of the file must be given as the key).

Example:

http {
    init_by_lua_block {
        local lua_resty_waf = require "resty.waf"

        -- this translates and calculates a ruleset called 'ruleset_name'
        local ok, errs = pcall(function()
            lua_resty_waf.load_secrules("/path/to/secrules/ruleset_name")
        end)

        -- errs is an array-like table
        if errs then
            for i = 1, #errs do
                ngx.log(ngx.ERR, errs[i])
            end
        end
    }

    server {
        location / {
            access_by_lua_block {
                local lua_resty_waf = require "resty.waf"

                local waf = lua_resty_waf:new()

                -- in order to use the loaded ruleset, it must be added via
                -- the 'add_ruleset' option
                waf:set_option("add_ruleset", "ruleset_name")
            }
        }
    }
}

Additionally, load_secrules can take an optional second argument as a table of options to pass to various translation functions. The following options are recognized:

  • path: Define a filesystem path to search for data files for operators such as @pmFromFile. If no such key is defined, the current working directory (.) is used
  • force: Do not error and abort when failing to translate a rule variable
  • loose: Do not error and abort when failing to translate a rule action
  • quiet: Do not error or warn when failing to translate a rule action

This function can also take a third option as a table to catch translation errors, for later processing. If this option is not present or a not a table, translation errors will instead be logged to the error log.

lua-resty-waf.init()

Perform some pre-computation of rules and rulesets, based on what's been made available via the default distributed rulesets. It's recommended, but not required, to call this function (not doing so will result in a small performance penalty). This function should never be called outside this scope.

Example:

http {
    init_by_lua_block {
        local lua_resty_waf = require "resty.waf"

        lua_resty_waf.init()
    }
}

Public Methods

lua-resty-waf:new()

Instantiate a new instance of lua-resty-waf. You must call this in every request handler phase you wish to run lua-resty-waf, and use the return result to call further object methods.

Example:

location / {
    access_by_lua_block {
        local lua_resty_waf = require "resty.waf"

        local waf = lua_resty_waf:new()
    }
}

lua-resty-waf:set_option()

Configure an option on a per-scope basis.

Example:

location / {
    access_by_lua_block {
        local lua_resty_waf = require "resty.waf"

        local waf = lua_resty_waf:new()

        -- enable debug logging only for this scope
        waf:set_option("debug", true)
    }
}

lua-resty-waf:set_var()

Define a transaction variable (stored in the TX variable collection) before executing the WAF. This can be used to define variables used by complex rulesets such as the OWASP CRS.

Example:

location / {
    access_by_lua_block {
        local lua_resty_waf = require "resty.waf"

        local waf = lua_resty_waf:new()

        waf:set_var("FOO", "bar")
    }
}

Note that as with any other ModSecurity rule, the existence of a variable bears no functional change to WAF processing; it is the responsibility of the rule author to understand and use TX variables.

lua-resty-waf:sieve_rule()

Define a collection exclusion for a given rule.

Example:

location / {
    access_by_lua_block {
        local lua_resty_waf = require "resty.waf"

        local waf = lua_resty_waf:new()

        local sieves = {
            {
                type   = "ARGS",
                elts   = "foo",
                action = "ignore",
            }
        }

        waf:sieve_rule("12345", sieves)
    }
}

See the rule sieves wiki page for details and advanced usage examples.

lua-resty-waf:exec()

Run the rule engine. By default, the engine is executed according to the currently running phase. An optional table may be passed, allowing users to "mock" execution of a different phase.

Example:

location / {
    access_by_lua_block {
        local lua_resty_waf = require "resty.waf"

        local waf = lua_resty_waf:new()

        -- execute according to access phase collections and rules
        waf:exec()
    }

    content_by_lua_block {
        local lua_resty_waf = require "waf"

        local waf = lua_resty_waf:new()

        -- execute header_filter rules, passing in a table of additional collections
        -- this assumes the 'request_headers' and 'status' Lua variables were
        -- declared and initialized elsewhere
        local opts = {
            phase = 'header_filter',
            collections = {
                REQUEST_HEADERS = request_headers,
                STATUS = status,
            }
        }

        waf:exec(opts)
    }
}

lua-resty-waf:write_log_events()

Write any audit log entries that were generated from the transaction. This is only optional when exec is called in a log_by_lua handler.

Example:

location / {
    log_by_lua_block {
        local lua_resty_waf = require "resty.waf"

        local waf = lua_resty_waf:new()

        -- write out any event log entries to the
        -- configured target, if applicable
        waf:write_log_events()
    }
}

Options

add_ruleset

Default: none

Adds an additional ruleset to be used during processing. This allows users to implement custom rulesets without stomping over the included rules directory. Additional rulesets must reside within a folder called "rules" that lives within the lua_package_path.

Example:

http {
    -- the rule file 50000.json must live at
    -- /path/to/extra/rulesets/rules/50000.json
    lua_package_path '/path/to/extra/rulesets/?.lua;;';

    server {
        location / {
            access_by_lua_block {
                waf:set_option("add_ruleset", "50000_extra_rules")
            }
        }
    }
}

Multiple rulesets may be added by passing a table of values to set_option. Note that ruleset names are sorted before processing. Rulesets are processed in a low-to-high sorted order.

add_ruleset_string

Default: none

Adds an additional ruleset to be used during processing. This allows users to implement custom rulesets without stomping over the included rules directory. Rulesets are defined inline as a Lua string, in the form of a translated ruleset JSON structure.

Example:

location / {
    access_by_lua_block {
        waf:set_option("add_ruleset_string", "70000_extra_rules", [=[{"access":[{"action":"DENY","id":73,"operator":"REGEX","opts":{},"pattern":"foo","vars":[{"parse":{"values":1},"type":"REQUEST_ARGS"}]}],"body_filter":[],"header_filter":[]}]=])
    }
}

Note that ruleset names are sorted before processing, and must be given as strings. Rulesets are processed in a low-to-high sorted order.

allow_unknown_content_types

Default: false

Instructs lua-resty-waf to continue processing the request when a Content-Type header has been sent that is not in the allowed_content_types table. Such requests will not have their request body processed by lua-resty-waf (the REQUEST_BODY collection will be nil). In this manner, users do not need to explicitly whitelist all possible Content-Type headers they may encounter.

Example:

location / {
    access_by_lua_block {
        waf:set_option("allow_unknown_content_types", true)
    }
}

allowed_content_types

Default: none

Defines one or more Content-Type headers that will be allowed, in addition to the default Content-Types application/x-www-form-urlencoded and multipart/form-data. A request whose content type matches one of allowed_content_types will set the REQUEST_BODY collection to a single string containing (rather than a table); a request whose content type does not match one of these values, or application/x-www-form-urlencoded or multipart/form-data, will be rejected.

Example:

location / {
    access_by_lua_block {
        -- define a single allowed Content-Type value
        waf:set_option("allowed_content_types", "text/xml")

        -- defines multiple allowed Content-Type values
        waf:set_option("allowed_content_types", { "text/html", "text/json", "application/json" })
    }
}

Note that mutiple set_option calls with a parameter of allowed_content_types will simply override the existing options table, so if you want to define multiple allowed content types, you must define them as a Lua table as shown above.

debug

Default: false

Disables/enables debug logging. Debug log statements are printed to the error_log. Note that debug logging is very expensive and should not be used in production environments.

Example:

location / {
    access_by_lua_block {
        waf:set_option("debug", true)
    }
}

debug_log_level

Default: ngx.INFO

Sets the nginx log level constant used for debug logging.

Example:

location / {
    access_by_lua_block {
        waf:set_option("debug_log_level", ngx.DEBUG)
    }
}

deny_status

Default: ngx.HTTP_FORBIDDEN

Sets the status to use when denying requests.

Example:

location / {
    access_by_lua_block {
        waf:set_option("deny_status", ngx.HTTP_NOT_FOUND)
    }
}

disable_pcre_optimization

Default: false

Removes the oj flags from all ngx.re.match, ngx.re.find, and ngx.re.sub calls. This may be useful in some cases where older PCRE libraries are used, but will cause severe performance degradation, so its use is strongly discouraged; users are instead encouraged to build OpenResty with a modern, JIT-capable PCRE library.

Example:

location / {
    access_by_lua_block {
        waf:set_option("disable_pcre_optimization", true)
    }
}

Note: This behavior is deprecated and will be removed in future versions.

event_log_altered_only

Default: true

Determines whether to write log entries for rule matches in a transaction that was not altered by lua-resty-waf. "Altered" is defined as lua-resty-waf acting on a rule whose action is ACCEPT or DENY. When this option is unset, lua-resty-waf will log rule matches even if the transaction was not altered. By default, lua-resty-waf will only write log entries for matches if the transaction was altered.

Example:

location / {
    access_by_lua_block {
        waf:set_option("event_log_altered_only", false)
    }
}

Note that mode will not have an effect on determing whether a transaction is considered altered. That is, if a rule with a DENY action is matched, but lua-resty-waf is running in SIMULATE mode, the transaction will still be considered altered, and rule matches will be logged.

event_log_buffer_size

Default: 4096

Defines the threshold size, in bytes, of the buffer to be used to hold event logs. The buffer will be flushed when this threshold is met.

Example:

location / {
    access_by_lua_block {
        -- 8 KB event log message buffer
        waf:set_option("event_log_buffer_size", 8192)
    }
}

热门推荐
阅读排行榜

扫描微信二维码

查看手机版网站

随时了解更新最新资讯

139-2527-9053

在线客服(服务时间 9:00~18:00)

在线QQ客服
地址:深圳市南山区西丽大学城创智工业园
电邮:jeky_zhao#qq.com
移动电话:139-2527-9053

Powered by 互联科技 X3.4© 2001-2213 极客世界.|Sitemap