The Programming Language Lua

&

OpenResty

            rmingwang@gmail.com
        

Kernel

Configuration

复杂软件系统

核心功能和内部实现,提供调用配置接口
  • 编译型语言
  • 静态语言
  • 注重性能
调用和配置 Kernel ,丰富和定制系统
  • 脚本语言
  • 动态语言
  • 注重业务逻辑

编程语言排行榜

# 2011年6月

🔽

编程语言排行榜

# 2015年8月

Why Lua?

  • 语法简单,代码可读性强
  • 完整解析器不到200K,可嵌入宿主程序
  • 最快的脚本语言,JIT,协程,同步书写,异步执行
  • 跨平台 ASCII C实现 (C API)
  • 自动内存管理(GC 标记清除法)

Lua is a powerful, fast, lightweight, embeddable scripting language.

🔽

* 接口语言 、嵌入式语言、胶水语言  

解析器对比

使用lua开发的游戏

- 魔兽争霸
- 梦幻西游
- 愤怒的小鸟(Wax)
...

API接口插件

- 魔兽世界(自定义皮肤界面)
- Dota2 (RPG地图编辑器脚本)
...

其他应用

- MySQL Proxy Scripting
- Wireshark Lua API 
- OpenWrt Lua UCI API (LuCI)
- Adobe Lightroom

WEB框架

- Sailor! A Lua MVC Framework
- Lapis (Web Framework OpenResty && Lua)
- aLiLua | A High-Performance Lua Web Framework

Isomorphic javascript


=> run on both the client and the server

Isomorphic Lua ?

"Write once, run anywhere" (WORA)

Java JVM(JIT inside)
JavaScript  V8 engine(JIT inside)
Lua Embedded / 200K / JIT / Ascii C  

C++, PHP, python...

x86/x64, ARM, MIPS, PPC, even => STM32, C51...

🔽

<?php
   $lua = new Plua($lua_script_file = NULL);
 
   $lua->eval("lua_statements");     //eval lua codes
   $lua->include("lua_script_file"); //import a lua script
 
   $lua->assign("name", $value); //assign a php variable to Lua
   $lua->register("name", $function); //register a PHP function to Lua with "name"
 
   $lua->call($string_lua_function_name, array() /*args*/);
   $lua->call($resouce_lua_anonymous_function, array() /*args);
   $lua->call(array("table", "method"), array(...., "push_self" => [true | false]) /*args*/);
 
   $lua->{$lua_function}(array()/*args*/);
?>
  

# Plua

  • INSTALL

  • Package Manager

  • Basic Usage

Lua language

print "input your name:"

local name = io.read()

print("hello "..name)
--print(string.format("hello %s", name))

# CLI 

Lua language

local person = {
    first = "rming",
    last  = "wang",
}

local _dump
_dump = function(t)
    for k,v in pairs(t) do
        if k ~= '_G' and type(v) == 'table' then
            _dump(v)
        else
            print(string.format('%s=>%s', k, v))
        end
    end    
end

_dump(person)

# table dump

Lua language

window = {}
window.prototype = {
    x = 10,
    y = 11,
    width  = 100,
    height = 50,
    print  = print,
}
-- 定义 metatable 并定义 __index 方法
window.mt = {}
--window.mt.__index = window.prototype
window.mt.__index = function(table, key)
    return window.prototype[key];   
end
-- 设置 window 元表
setmetatable(window, window.mt)

-- 定义 window 构造函数
window.new = function(o)
    setmetatable(o, window.mt)
    return o
end

new_window = window.new{x = 20, y = 21}
--new_window = window.new({x = 20, y = 21})
--new_window = window.new{}

--[[
    new_window -> 有没有 x key --没有-> metatable.__index -> 返回 window.prototype[x]
--]]

new_window.print 'hello'
print(new_window.x)
print(new_window.width)

# CLASS 

Lua language

  • 默认全局变量
  • 匿名函数
  • nil false 假
  • 函数一等公民
  • 元表 <=> 原型
  • 标准库
  • ...

 

 OpenResty = Nginx core + 3rd-party Nginx modules

Why Nginx?

  • I/O multiplexing
  • Connections Pool
  • noBlocking!

 

" Don't call me, I will call you. "

nginx.conf scripting

- nginx + perl ?  

    ngx_http_perl_module  

- nginx + javascript ?  
    *ngx_http_js_module  

- nginx + lua ?  
    ngx_http_lua_module  

nginx.conf scripting

- Lua + Nginx API for Lua

          + ngx.shared.DICT

          + memcache / redis / mysql / postgresql 

 

- 应用  
         - 防止DDos,清洗流量  
         - XSS filter  
         - upstream节点健康检查  
         - 图片处理    
         - ...

 

nginx.conf scripting

server
{
        listen 80;
        server_name openresty.dev;

        #use ngx_echo_module
        location = '/halo' {
                add_header Content-Type text/html;
                echo "halo, $arg_person!";
                break;
        }

        #use ngx_http_lua_module
        location ~* ^/test(.*)$ {
            #lua_code_cache off;
            content_by_lua "
                ngx.send_headers({'Content-Type:text/html'})
                ngx.say('tester')
                ngx.exit(200)
            ";
        }

}

# halo word

🔽

nginx.conf scripting

http://openresty.dev/halo?person=rmingwang
halo, rmingwang!


http://openresty.dev/test
tester

# halo word

nginx.conf scripting

🔽

nginx.conf scripting

#Nginx请求处理流程 和 Lua处理调用

 

nginx.conf scripting

NGX_HTTP_POST_READ_PHASE:
    #读取请求内容阶段

NGX_HTTP_SERVER_REWRITE_PHASE:
    #Server请求地址重写阶段

NGX_HTTP_FIND_CONFIG_PHASE:
    #配置查找阶段:

NGX_HTTP_REWRITE_PHASE:
    #Location请求地址重写阶段

NGX_HTTP_POST_REWRITE_PHASE:
    #请求地址重写提交阶段

NGX_HTTP_PREACCESS_PHASE:
    #访问权限检查准备阶段

NGX_HTTP_ACCESS_PHASE:
    #访问权限检查阶段

NGX_HTTP_POST_ACCESS_PHASE:
    #访问权限检查提交阶段

NGX_HTTP_TRY_FILES_PHASE:
    #配置项try_files处理阶段

NGX_HTTP_CONTENT_PHASE:
    #内容产生阶段

NGX_HTTP_LOG_PHASE:
    #日志模块处理阶段

# nginx请求处理流程

nginx.conf scripting

init_by_lua:
在nginx重新加载配置文件时,运行里面lua脚本,常用于全局变量的申请。
例如lua_shared_dict共享内存的申请,只有当nginx重起后,共享内存数据才清空,这常用于统计。

set_by_lua:
设置一个变量,常用与计算一个逻辑,然后返回结果
该阶段不能运行Output API、Control API、Subrequest API、Cosocket API

rewrite_by_lua:
在access阶段前运行,主要用于rewrite

access_by_lua:
主要用于访问控制,能收集到大部分变量,类似status需要在log阶段才有。
这条指令运行于nginx access阶段的末尾,因此总是在 allow 和 deny 这样的指令之后运行,虽然它们同属 access 阶段。

content_by_lua:
阶段是所有请求处理阶段中最为重要的一个,运行在这个阶段的配置指令一般都肩负着生成内容(content)并输出HTTP响应。

header_filter_by_lua:
一般只用于设置Cookie和Headers等
该阶段不能运行Output API、Control API、Subrequest API、Cosocket API

body_filter_by_lua:
一般会在一次请求中被调用多次, 因为这是实现基于 HTTP 1.1 chunked 编码的所谓“流式输出”的。
该阶段不能运行Output API、Control API、Subrequest API、Cosocket API

log_by_lua:
该阶段总是运行在请求结束的时候,用于请求的后续操作,如在共享内存中进行统计数据,如果要高精确的数据统计,应该使用body_filter_by_lua。
该阶段不能运行Output API、Control API、Subrequest API、Cosocket API

nginx.conf scripting

init_by_lua            http
set_by_lua             server, server if, location, location if
rewrite_by_lua         http, server, location, location if
access_by_lua          http, server, location, location if
content_by_lua         location, location if
header_filter_by_lua   http, server, location, location if
body_filter_by_lua     http, server, location, location if
log_by_lua             http, server, location, location if
timer
# Nginx下Lua处理阶段与使用范围

配置文件『声明性的』,而非『过程性的』

nginx.conf scripting

nginx.conf scripting

-- 发起同步非阻塞请求
ngx.location.capture("/foo") 
ngx.location.capture_multi{ {"/foo"}, "/bar"}}

-- 数据字典操作,可以在nginx不同 worker 的 pool 间共享
--lua_shared_dict dogs 10m;
local dogs = ngx.shared.dogs 
dogs:add("Tom", 5) 
dogs:set("Tom", 7) 
dogs:incr("Tom", 1) 
dogs:delete("Tom", 1)

-- 延时执行
local function do_job() 
    ...
end
local ok, err = ngx.timer.at(1.5, do_job)

nginx.conf scripting

# Construct fully RESTful queries in a single location


location ~ '^/cat/(\d+)' {
    set $id $1;
    set_form_input $name; set_quote_sql_str $quoted_name $name;

    postgres_query GET "select * from cats where id=$id";
    postgres_query DELETE "delete from cats where id=$id";
    postgres_query POST "insert into cats (id, name) values($id, $quoted_name)";

    postgres_pass my_pg_backend; 
}

non-blocking MySQL

http {
    ...

    upstream cluster {
        # simple round-robin
        drizzle_server 127.0.0.1:3306 dbname=test
             password=some_pass user=monty protocol=mysql;
        drizzle_server 127.0.0.1:1234 dbname=test2
             password=pass user=bob protocol=drizzle;
    }

    upstream backend {
        drizzle_server 127.0.0.1:3306 dbname=test
             password=some_pass user=monty protocol=mysql;
    }

    server {
        location /mysql {
            drizzle_query $query_string;

            drizzle_pass backend;

            drizzle_connect_timeout    500ms; # default 60s
            drizzle_send_query_timeout 2s;    # default 60s
            drizzle_recv_cols_timeout  1s;    # default 60s
            drizzle_recv_rows_timeout  1s;    # default 60s
        }

        ...

        # for connection pool monitoring
        location /mysql-pool-status {
            allow 127.0.0.1;
            deny all;

            drizzle_status;
        }
    }
}
nginx.conf scripting

non-blocking MySQL

# php-fpm ->mysql :blocked mysql connection 

nginx.conf scripting

🔽

non-blocking MySQL

# php-fpm ->mysql :blocked mysql connection 

nginx.conf scripting

non-blocking MySQL

nginx.conf scripting

# nginx -> libdrizzle -> mysql connection

🔽

non-blocking MySQL

# nginx -> libdrizzle -> mysql connection

nginx.conf scripting

non-blocking Service

nginx.conf scripting

🔽

non-blocking Service

nginx.conf scripting

non-blocking Service

nginx.conf scripting

non-blocking Service

nginx.conf scripting

nginx lua access control

nginx.conf scripting
local function args(key, default)
    local args = ngx.req.get_uri_args()
    return args[key] or default
end


local function set_header(header)
    ngx.header.content_type = header;
end

local function pairsByKeys(t)
    local a = {}
    for n in pairs(t) do a[#a + 1] = n end
    table.sort(a)
    local i = 0
    return function ()
        i = i + 1
        return a[i], t[a[i]]
    end
end


local function is_expired(token_config)
    -- if token.date expired
    token_time = args('date', 0)
    return token_config['expires'] and os.date('%Y-%m-%d') ~= args('date') or false
end


local function is_full_params()
    -- if full params
    local tokens = {args('date'), args('version'), args('sign')}
    local count  = 0
    for k,v in pairs(tokens) do
        count =count+1
    end
    return count==3
end

local function is_verified(token_config)
    -- if token verified
    local token_str = ''
    local url = ngx.var.scheme..'://'..ngx.var.http_host..ngx.var.uri
    local tokens = {ngx.md5(url), args('date'), args('version'), token_config['secret']}
    for k,v in pairsByKeys(tokens) do
        token_str = token_str..v
    end
    return args('sign') == ngx.md5(token_str)
end

--[[
    set config['expires'] = false disable token expires
]]--
local config = {
    secret  = 'your_secret_here',
    expires = true,
}

if not is_full_params() or is_expired(config) or not is_verified(config)  then
    ngx.exit(ngx.HTTP_FORBIDDEN)
end

nginx lua random avatar

nginx.conf scripting
local function args(key, default)
    local args = ngx.req.get_uri_args()
    return args[key] or default
end


local function set_header(header)
    ngx.header.content_type = header;
end

local function pairsByKeys(t)
    local a = {}
    for n in pairs(t) do a[#a + 1] = n end
    table.sort(a)
    local i = 0
    return function ()
        i = i + 1
        return a[i], t[a[i]]
    end
end

function split(inputstr, sep)
    if sep == nil then
        sep = "%s"
    end
    local t={} ; i=1
    for str in string.gmatch(inputstr, "([^"..sep.."]+)") do
        t[i] = str
        i = i + 1
    end
    return t
end


local function is_full_params()
    -- if full params
    local params = {
        appid = args('appid'),
        pid   = args('pid'),
        uid   = args('uid'),
        v     = args('v')
    }
    local count  = 0
    for k,v in pairs(params) do
        count = count + 1
    end
    return count==4 and params or false
end

local function range(min, max)
    local rangex = {}
    for i = min, max do
        table.insert(rangex, i)
    end
    return rangex
end

function file_exists(name)
   local f=io.open(name,"r")
   if f~=nil then io.close(f) return true else return false end
end

local function redis_connect(address, port)
    local redis = require "resty.redis"
    local red = redis:new()
    red:set_timeout(1000)

    local ok, err = red:connect(address, port)
    if not ok then
        ngx.log(ngx.ERR, "failed to connect: ", err)
        return false
    else
        return red
    end
end


local function generateCMD(gmbin, color, origin, dst, dst_width, dst_height)
    local dst_width  = dst_width or 120
    local dst_height = dst_height or 120
    local cmd = '%s convert -background "%s" -gravity center -extent %dx%d %s %s'
    return string.format(cmd, gmbin, color, dst_width, dst_height, origin, dst)
end

--局部函数,防止lua调用全局函数报错
local random_avatar,find_avatar,random
random = require "resty.random"

--从查询头像池中取得一个头像
random_avatar = function (red, post_hash_k, user_filed, root_filed, avatars_pool, colors)
    --spop query [avatar set remained]
    local res, err = red:spop(avatars_pool)
    local error_message = function(format)
        return string.format(format, avatars_pool)
    end
    if not res then
        --redis query error
        ngx.log(ngx.ERR, error_message("failed spop %s: "), err)
    elseif res == ngx.null then
        --avatar pool is empty
        ngx.log(ngx.INFO, error_message("%s pool is empty "), err)
        --查询楼主头像,产生头像池
        local resx = find_avatar(red, post_hash_k, root_filed, root_filed, avatars_pool, colors)
        if not resx then return false end
        for i,v in pairs(range(1, 12)) do
            if v~=resx then
                red:sadd(avatars_pool, v)
            end
        end

        --递归调用继续查询随机头像
        res = random_avatar(red, post_hash_k, user_filed, root_filed, avatars_pool, colors)
    else
        --random avatar found
        ngx.log(ngx.INFO, error_message("spop %s: "), res)
        --随机生成后,记录到redis
        res = string.format('%s|%s', res, colors[random.number(1,12)])
        red:hset(post_hash_k, user_filed, res)
    end
    return res
end


--是否存在头像记录
find_avatar = function (red, post_hash_k, user_filed, root_filed, avatars_pool, colors)
    --hget query [hash user_filed]
    local res, err = red:hget(post_hash_k, user_filed)
    --error_message closure
    local error_message = function(format)
        return string.format(format, post_hash_k, user_filed)
    end
    if not res then
        --redis query error
        ngx.log(ngx.ERR, error_message("failed hmget %s %s: "), err)
    elseif res == ngx.null then
        --hash filed not found
        ngx.log(ngx.INFO, error_message("%s %s not found "), err)
        if user_filed==root_filed then
            --如果找不到楼主什么头像,则生成一个头像
            res = random.number(1,12)
            res = string.format('%s|%s', res, colors[random.number(1,12)])
            red:hset(post_hash_k, root_filed, res)
        else
            --找不到头像时,头像池中取一个头像
            res = random_avatar(red, post_hash_k, user_filed, root_filed, avatars_pool, colors)
        end
    else
        --hash filed found
        ngx.log(ngx.INFO, error_message("hget %s %s: "), res)
    end
    return res
end

--http://file.avatar.com/avatar?appid=sapp&pid=11&uid=1&v=2
set_header('text/plain')

--检查参数是否完整
local params = is_full_params()
if not params then
    ngx.exit(ngx.HTTP_NOT_FOUND)
end

--定义 redis key 的规则
local post_hash_k  = string.format('avatar.%s.%s.%s', params['appid'], params['pid'], params['v'])
local avatars_pool = string.format('%s.pool', post_hash_k)
local user_filed   = string.format('u.%s', params['uid'])
local root_filed   = string.format('u.%s', 0)
--local avatar_file = string.format('avatar.file.%s.%s', avatar_id, color_id)

local config
if  file_exists(ngx.var.document_root..'/online') then
    config = {
        --网站目录相关配置
        webroot = ngx.var.document_root,
        origin  = '/static/avatars/%d.png',
        dst     = '/upload/avatars/thumbnails/%s.png',
        --GraphicsMagick 路径
        gmbin   = '/usr/bin/gm',
        --redis 配置
        redis = {
            host = '10.172.169.246',
            port = 6379,
        }
    }
elseif file_exists(ngx.var.document_root..'/test') then
    config = {
        --网站目录相关配置
        webroot = ngx.var.document_root,
        origin  = '/static/avatars/%d.png',
        dst     = '/upload/avatars/thumbnails/%s.png',
        --GraphicsMagick 路径
        gmbin   = '/usr/bin/gm',
        --redis 配置
        redis = {
            host = '10.171.80.244',
            port = 6379,
        }
    }
else
    config = {
        --网站目录相关配置
        webroot = ngx.var.document_root,
        origin  = '/static/avatars/%d.png',
        dst     = '/upload/avatars/thumbnails/%s.png',
        --GraphicsMagick 路径
        gmbin   = '/usr/local/bin/gm',
        --redis 配置
        redis = {
            host = '127.0.0.1',
            port = 6379,
        }
    }
end

local colors  = {'#fd788e', '#ffcd43', '#77b7fc', '#9f90db', '#60d5b5', '#ff8784', '#e5619d', '#00c2de', '#d19dc6', '#4ee3f0', '#00a4df', '#80dd80'}
local avatars = range(1,12)

--redis connection
local red = redis_connect(config.redis.host, config.redis.port)
if not red then return end

local avatar_data = find_avatar(red, post_hash_k, user_filed, root_filed, avatars_pool, colors)
if not avatar_data then return end

local avatar_data = split(avatar_data, '|')
local color       = table.remove(avatar_data) or colors[random.number(1,12)]
local avatar_id   = table.remove(avatar_data) or random.number(1,12)

local dst_uri = string.format(config.dst, avatar_id..color)
local origin  = config.webroot..string.format(config.origin, avatar_id)
local dst     = config.webroot..dst_uri

local cmd = generateCMD(config.gmbin, color, origin, dst)

if file_exists(dst) then
    ngx.exec(dst_uri)
else
    _,_,dst_dir,dst_filename = string.find(dst,'(.-)([^/]*)$')
    os.execute('mkdir -p '..dst_dir)
    os.execute(cmd)
    if file_exists(dst) then
        ngx.exec(dst_uri)
    else
        ngx.exit(ngx.ERR)
    end
end

Any questions?

Thank you!

* 图片、代码来源于网络,感谢作者分享