The Programming Language Lua
&
OpenResty
rmingwang@gmail.com
Kernel
Configuration
复杂软件系统
核心功能和内部实现,提供调用配置接口
-
编译型语言
- 静态语言
- 注重性能
调用和配置 Kernel ,丰富和定制系统
-
脚本语言
- 动态语言
- 注重业务逻辑
编程语言排行榜
# 2011年6月
🔽
编程语言排行榜
# 2015年8月
Why Lua?
- 语法简单,代码可读性强
- 完整解析器不到200K,可嵌入宿主程序
- 最快的脚本语言,JIT,协程,同步书写,异步执行
- 跨平台 ASCII C实现 (C API)
- 自动内存管理(GC 标记清除法)
Lua is a powerful, fast, lightweight, embeddable scripting language.
🔽
* 接口语言 、嵌入式语言、胶水语言
解析器对比
使用lua开发的游戏
- 魔兽争霸
- 梦幻西游
- 愤怒的小鸟(Wax)
...
API接口插件
- 魔兽世界(自定义皮肤界面)
- Dota2 (RPG地图编辑器脚本)
...
其他应用
- MySQL Proxy Scripting
- Wireshark Lua API
- OpenWrt Lua UCI API (LuCI)
- Adobe Lightroom
WEB框架
- Sailor! A Lua MVC Framework
- Lapis (Web Framework OpenResty && Lua)
- aLiLua | A High-Performance Lua Web Framework
Isomorphic javascript
=> run on both the client and the server
Isomorphic Lua ?
"Write once, run anywhere" (WORA)
Java | JVM(JIT inside) |
JavaScript | V8 engine(JIT inside) |
Lua | Embedded / 200K / JIT / Ascii C |
C++, PHP, python...
x86/x64, ARM, MIPS, PPC, even => STM32, C51...
🔽
<?php
$lua = new Plua($lua_script_file = NULL);
$lua->eval("lua_statements"); //eval lua codes
$lua->include("lua_script_file"); //import a lua script
$lua->assign("name", $value); //assign a php variable to Lua
$lua->register("name", $function); //register a PHP function to Lua with "name"
$lua->call($string_lua_function_name, array() /*args*/);
$lua->call($resouce_lua_anonymous_function, array() /*args);
$lua->call(array("table", "method"), array(...., "push_self" => [true | false]) /*args*/);
$lua->{$lua_function}(array()/*args*/);
?>
# Plua
-
INSTALL
-
Package Manager
-
Basic Usage
- Lua简明教程
- Lua程序设计
Lua language
print "input your name:"
local name = io.read()
print("hello "..name)
--print(string.format("hello %s", name))
# CLI
Lua language
local person = {
first = "rming",
last = "wang",
}
local _dump
_dump = function(t)
for k,v in pairs(t) do
if k ~= '_G' and type(v) == 'table' then
_dump(v)
else
print(string.format('%s=>%s', k, v))
end
end
end
_dump(person)
# table dump
Lua language
window = {}
window.prototype = {
x = 10,
y = 11,
width = 100,
height = 50,
print = print,
}
-- 定义 metatable 并定义 __index 方法
window.mt = {}
--window.mt.__index = window.prototype
window.mt.__index = function(table, key)
return window.prototype[key];
end
-- 设置 window 元表
setmetatable(window, window.mt)
-- 定义 window 构造函数
window.new = function(o)
setmetatable(o, window.mt)
return o
end
new_window = window.new{x = 20, y = 21}
--new_window = window.new({x = 20, y = 21})
--new_window = window.new{}
--[[
new_window -> 有没有 x key --没有-> metatable.__index -> 返回 window.prototype[x]
--]]
new_window.print 'hello'
print(new_window.x)
print(new_window.width)
# CLASS
Lua language
- 默认全局变量
- 匿名函数
- nil false 假
- 函数一等公民
- 元表 <=> 原型
- 标准库
- ...
OpenResty = Nginx core + 3rd-party Nginx modules
- LuaJIT (Just-In-Time Compiler for the Lua programming language. )
- HttpLuaModule (ngx_lua - Embed the power of Lua into Nginx)
- LuaRestyCoreLibrary (lua-resty-core - New FFI-based Lua API for the ngx_lua module)
- ...
Why Nginx?
- I/O multiplexing
- Connections Pool
- noBlocking!
" Don't call me, I will call you. "
-
INSTALL
-
3rdPartyModules
-
Basic Usage
nginx.conf scripting
- nginx + perl ?
- nginx + javascript ?
*ngx_http_js_module
- nginx + lua ?
ngx_http_lua_module
nginx.conf scripting
- Lua + Nginx API for Lua
+ ngx.shared.DICT
+ memcache / redis / mysql / postgresql
- 应用
- 防止DDos,清洗流量
- XSS filter
- upstream节点健康检查
- 图片处理
- ...
nginx.conf scripting
server
{
listen 80;
server_name openresty.dev;
#use ngx_echo_module
location = '/halo' {
add_header Content-Type text/html;
echo "halo, $arg_person!";
break;
}
#use ngx_http_lua_module
location ~* ^/test(.*)$ {
#lua_code_cache off;
content_by_lua "
ngx.send_headers({'Content-Type:text/html'})
ngx.say('tester')
ngx.exit(200)
";
}
}
# halo word
🔽
nginx.conf scripting
http://openresty.dev/halo?person=rmingwang
halo, rmingwang!
http://openresty.dev/test
tester
# halo word
nginx.conf scripting
🔽
nginx.conf scripting
#Nginx请求处理流程 和 Lua处理调用
nginx.conf scripting
NGX_HTTP_POST_READ_PHASE:
#读取请求内容阶段
NGX_HTTP_SERVER_REWRITE_PHASE:
#Server请求地址重写阶段
NGX_HTTP_FIND_CONFIG_PHASE:
#配置查找阶段:
NGX_HTTP_REWRITE_PHASE:
#Location请求地址重写阶段
NGX_HTTP_POST_REWRITE_PHASE:
#请求地址重写提交阶段
NGX_HTTP_PREACCESS_PHASE:
#访问权限检查准备阶段
NGX_HTTP_ACCESS_PHASE:
#访问权限检查阶段
NGX_HTTP_POST_ACCESS_PHASE:
#访问权限检查提交阶段
NGX_HTTP_TRY_FILES_PHASE:
#配置项try_files处理阶段
NGX_HTTP_CONTENT_PHASE:
#内容产生阶段
NGX_HTTP_LOG_PHASE:
#日志模块处理阶段
# nginx请求处理流程
nginx.conf scripting
init_by_lua:
在nginx重新加载配置文件时,运行里面lua脚本,常用于全局变量的申请。
例如lua_shared_dict共享内存的申请,只有当nginx重起后,共享内存数据才清空,这常用于统计。
set_by_lua:
设置一个变量,常用与计算一个逻辑,然后返回结果
该阶段不能运行Output API、Control API、Subrequest API、Cosocket API
rewrite_by_lua:
在access阶段前运行,主要用于rewrite
access_by_lua:
主要用于访问控制,能收集到大部分变量,类似status需要在log阶段才有。
这条指令运行于nginx access阶段的末尾,因此总是在 allow 和 deny 这样的指令之后运行,虽然它们同属 access 阶段。
content_by_lua:
阶段是所有请求处理阶段中最为重要的一个,运行在这个阶段的配置指令一般都肩负着生成内容(content)并输出HTTP响应。
header_filter_by_lua:
一般只用于设置Cookie和Headers等
该阶段不能运行Output API、Control API、Subrequest API、Cosocket API
body_filter_by_lua:
一般会在一次请求中被调用多次, 因为这是实现基于 HTTP 1.1 chunked 编码的所谓“流式输出”的。
该阶段不能运行Output API、Control API、Subrequest API、Cosocket API
log_by_lua:
该阶段总是运行在请求结束的时候,用于请求的后续操作,如在共享内存中进行统计数据,如果要高精确的数据统计,应该使用body_filter_by_lua。
该阶段不能运行Output API、Control API、Subrequest API、Cosocket API
nginx.conf scripting
init_by_lua http
set_by_lua server, server if, location, location if
rewrite_by_lua http, server, location, location if
access_by_lua http, server, location, location if
content_by_lua location, location if
header_filter_by_lua http, server, location, location if
body_filter_by_lua http, server, location, location if
log_by_lua http, server, location, location if
timer
# Nginx下Lua处理阶段与使用范围
配置文件『声明性的』,而非『过程性的』
nginx.conf scripting
nginx.conf scripting
-- 发起同步非阻塞请求
ngx.location.capture("/foo")
ngx.location.capture_multi{ {"/foo"}, "/bar"}}
-- 数据字典操作,可以在nginx不同 worker 的 pool 间共享
--lua_shared_dict dogs 10m;
local dogs = ngx.shared.dogs
dogs:add("Tom", 5)
dogs:set("Tom", 7)
dogs:incr("Tom", 1)
dogs:delete("Tom", 1)
-- 延时执行
local function do_job()
...
end
local ok, err = ngx.timer.at(1.5, do_job)
nginx.conf scripting
# Construct fully RESTful queries in a single location
location ~ '^/cat/(\d+)' {
set $id $1;
set_form_input $name; set_quote_sql_str $quoted_name $name;
postgres_query GET "select * from cats where id=$id";
postgres_query DELETE "delete from cats where id=$id";
postgres_query POST "insert into cats (id, name) values($id, $quoted_name)";
postgres_pass my_pg_backend;
}
non-blocking MySQL
http {
...
upstream cluster {
# simple round-robin
drizzle_server 127.0.0.1:3306 dbname=test
password=some_pass user=monty protocol=mysql;
drizzle_server 127.0.0.1:1234 dbname=test2
password=pass user=bob protocol=drizzle;
}
upstream backend {
drizzle_server 127.0.0.1:3306 dbname=test
password=some_pass user=monty protocol=mysql;
}
server {
location /mysql {
drizzle_query $query_string;
drizzle_pass backend;
drizzle_connect_timeout 500ms; # default 60s
drizzle_send_query_timeout 2s; # default 60s
drizzle_recv_cols_timeout 1s; # default 60s
drizzle_recv_rows_timeout 1s; # default 60s
}
...
# for connection pool monitoring
location /mysql-pool-status {
allow 127.0.0.1;
deny all;
drizzle_status;
}
}
}
nginx.conf scripting
non-blocking MySQL
# php-fpm ->mysql :blocked mysql connection
nginx.conf scripting
🔽
non-blocking MySQL
# php-fpm ->mysql :blocked mysql connection
nginx.conf scripting
non-blocking MySQL
nginx.conf scripting
# nginx -> libdrizzle -> mysql connection
🔽
non-blocking MySQL
# nginx -> libdrizzle -> mysql connection
nginx.conf scripting
non-blocking Service
nginx.conf scripting
🔽
non-blocking Service
nginx.conf scripting
non-blocking Service
nginx.conf scripting
non-blocking Service
nginx.conf scripting
nginx lua access control
nginx.conf scripting
local function args(key, default)
local args = ngx.req.get_uri_args()
return args[key] or default
end
local function set_header(header)
ngx.header.content_type = header;
end
local function pairsByKeys(t)
local a = {}
for n in pairs(t) do a[#a + 1] = n end
table.sort(a)
local i = 0
return function ()
i = i + 1
return a[i], t[a[i]]
end
end
local function is_expired(token_config)
-- if token.date expired
token_time = args('date', 0)
return token_config['expires'] and os.date('%Y-%m-%d') ~= args('date') or false
end
local function is_full_params()
-- if full params
local tokens = {args('date'), args('version'), args('sign')}
local count = 0
for k,v in pairs(tokens) do
count =count+1
end
return count==3
end
local function is_verified(token_config)
-- if token verified
local token_str = ''
local url = ngx.var.scheme..'://'..ngx.var.http_host..ngx.var.uri
local tokens = {ngx.md5(url), args('date'), args('version'), token_config['secret']}
for k,v in pairsByKeys(tokens) do
token_str = token_str..v
end
return args('sign') == ngx.md5(token_str)
end
--[[
set config['expires'] = false disable token expires
]]--
local config = {
secret = 'your_secret_here',
expires = true,
}
if not is_full_params() or is_expired(config) or not is_verified(config) then
ngx.exit(ngx.HTTP_FORBIDDEN)
end
nginx lua random avatar
nginx.conf scripting
local function args(key, default)
local args = ngx.req.get_uri_args()
return args[key] or default
end
local function set_header(header)
ngx.header.content_type = header;
end
local function pairsByKeys(t)
local a = {}
for n in pairs(t) do a[#a + 1] = n end
table.sort(a)
local i = 0
return function ()
i = i + 1
return a[i], t[a[i]]
end
end
function split(inputstr, sep)
if sep == nil then
sep = "%s"
end
local t={} ; i=1
for str in string.gmatch(inputstr, "([^"..sep.."]+)") do
t[i] = str
i = i + 1
end
return t
end
local function is_full_params()
-- if full params
local params = {
appid = args('appid'),
pid = args('pid'),
uid = args('uid'),
v = args('v')
}
local count = 0
for k,v in pairs(params) do
count = count + 1
end
return count==4 and params or false
end
local function range(min, max)
local rangex = {}
for i = min, max do
table.insert(rangex, i)
end
return rangex
end
function file_exists(name)
local f=io.open(name,"r")
if f~=nil then io.close(f) return true else return false end
end
local function redis_connect(address, port)
local redis = require "resty.redis"
local red = redis:new()
red:set_timeout(1000)
local ok, err = red:connect(address, port)
if not ok then
ngx.log(ngx.ERR, "failed to connect: ", err)
return false
else
return red
end
end
local function generateCMD(gmbin, color, origin, dst, dst_width, dst_height)
local dst_width = dst_width or 120
local dst_height = dst_height or 120
local cmd = '%s convert -background "%s" -gravity center -extent %dx%d %s %s'
return string.format(cmd, gmbin, color, dst_width, dst_height, origin, dst)
end
--局部函数,防止lua调用全局函数报错
local random_avatar,find_avatar,random
random = require "resty.random"
--从查询头像池中取得一个头像
random_avatar = function (red, post_hash_k, user_filed, root_filed, avatars_pool, colors)
--spop query [avatar set remained]
local res, err = red:spop(avatars_pool)
local error_message = function(format)
return string.format(format, avatars_pool)
end
if not res then
--redis query error
ngx.log(ngx.ERR, error_message("failed spop %s: "), err)
elseif res == ngx.null then
--avatar pool is empty
ngx.log(ngx.INFO, error_message("%s pool is empty "), err)
--查询楼主头像,产生头像池
local resx = find_avatar(red, post_hash_k, root_filed, root_filed, avatars_pool, colors)
if not resx then return false end
for i,v in pairs(range(1, 12)) do
if v~=resx then
red:sadd(avatars_pool, v)
end
end
--递归调用继续查询随机头像
res = random_avatar(red, post_hash_k, user_filed, root_filed, avatars_pool, colors)
else
--random avatar found
ngx.log(ngx.INFO, error_message("spop %s: "), res)
--随机生成后,记录到redis
res = string.format('%s|%s', res, colors[random.number(1,12)])
red:hset(post_hash_k, user_filed, res)
end
return res
end
--是否存在头像记录
find_avatar = function (red, post_hash_k, user_filed, root_filed, avatars_pool, colors)
--hget query [hash user_filed]
local res, err = red:hget(post_hash_k, user_filed)
--error_message closure
local error_message = function(format)
return string.format(format, post_hash_k, user_filed)
end
if not res then
--redis query error
ngx.log(ngx.ERR, error_message("failed hmget %s %s: "), err)
elseif res == ngx.null then
--hash filed not found
ngx.log(ngx.INFO, error_message("%s %s not found "), err)
if user_filed==root_filed then
--如果找不到楼主什么头像,则生成一个头像
res = random.number(1,12)
res = string.format('%s|%s', res, colors[random.number(1,12)])
red:hset(post_hash_k, root_filed, res)
else
--找不到头像时,头像池中取一个头像
res = random_avatar(red, post_hash_k, user_filed, root_filed, avatars_pool, colors)
end
else
--hash filed found
ngx.log(ngx.INFO, error_message("hget %s %s: "), res)
end
return res
end
--http://file.avatar.com/avatar?appid=sapp&pid=11&uid=1&v=2
set_header('text/plain')
--检查参数是否完整
local params = is_full_params()
if not params then
ngx.exit(ngx.HTTP_NOT_FOUND)
end
--定义 redis key 的规则
local post_hash_k = string.format('avatar.%s.%s.%s', params['appid'], params['pid'], params['v'])
local avatars_pool = string.format('%s.pool', post_hash_k)
local user_filed = string.format('u.%s', params['uid'])
local root_filed = string.format('u.%s', 0)
--local avatar_file = string.format('avatar.file.%s.%s', avatar_id, color_id)
local config
if file_exists(ngx.var.document_root..'/online') then
config = {
--网站目录相关配置
webroot = ngx.var.document_root,
origin = '/static/avatars/%d.png',
dst = '/upload/avatars/thumbnails/%s.png',
--GraphicsMagick 路径
gmbin = '/usr/bin/gm',
--redis 配置
redis = {
host = '10.172.169.246',
port = 6379,
}
}
elseif file_exists(ngx.var.document_root..'/test') then
config = {
--网站目录相关配置
webroot = ngx.var.document_root,
origin = '/static/avatars/%d.png',
dst = '/upload/avatars/thumbnails/%s.png',
--GraphicsMagick 路径
gmbin = '/usr/bin/gm',
--redis 配置
redis = {
host = '10.171.80.244',
port = 6379,
}
}
else
config = {
--网站目录相关配置
webroot = ngx.var.document_root,
origin = '/static/avatars/%d.png',
dst = '/upload/avatars/thumbnails/%s.png',
--GraphicsMagick 路径
gmbin = '/usr/local/bin/gm',
--redis 配置
redis = {
host = '127.0.0.1',
port = 6379,
}
}
end
local colors = {'#fd788e', '#ffcd43', '#77b7fc', '#9f90db', '#60d5b5', '#ff8784', '#e5619d', '#00c2de', '#d19dc6', '#4ee3f0', '#00a4df', '#80dd80'}
local avatars = range(1,12)
--redis connection
local red = redis_connect(config.redis.host, config.redis.port)
if not red then return end
local avatar_data = find_avatar(red, post_hash_k, user_filed, root_filed, avatars_pool, colors)
if not avatar_data then return end
local avatar_data = split(avatar_data, '|')
local color = table.remove(avatar_data) or colors[random.number(1,12)]
local avatar_id = table.remove(avatar_data) or random.number(1,12)
local dst_uri = string.format(config.dst, avatar_id..color)
local origin = config.webroot..string.format(config.origin, avatar_id)
local dst = config.webroot..dst_uri
local cmd = generateCMD(config.gmbin, color, origin, dst)
if file_exists(dst) then
ngx.exec(dst_uri)
else
_,_,dst_dir,dst_filename = string.find(dst,'(.-)([^/]*)$')
os.execute('mkdir -p '..dst_dir)
os.execute(cmd)
if file_exists(dst) then
ngx.exec(dst_uri)
else
ngx.exit(ngx.ERR)
end
end
Any questions?
Thank you!
* 图片、代码来源于网络,感谢作者分享
deck
By rmingwang
deck
- 5,156