Error: bad argument #1 to 'insert' (table expected, got nil) - mysql

I am trying to connect to a mysql server using LuaSql via a mysql proxy. I try to execute a simple program (db.lua):
require("luasql.mysql")
local _sqlEnv = assert(luasql.mysql())
local _con = nil
function read_auth(auth)
local host, port = string.match(proxy.backends[1].address, "(.*):(.*)")
_con = assert(_sqlEnv:connect( "db_name", "username", "password", "hostname", "3306"))
end
function disconnect_client()
assert(_con:close())
end
function read_query(packet)
local cur = con:execute("select * from t1")
myTable = {}
row = cur:fetch(myTable, "a")
print(myTable.id,myTable.user)
end
This code executes well when I execute it without mysql-proxy. When I am connecting with mysql-proxy, the error-log displays these errors:
mysql.lua:8: bad argument #1 to 'insert' (table expected, got nil)
db.lua:1: loop or previous error loading module 'luasql.mysql'
mysql.lua is a default file of LuaSql:
---------------------------------------------------------------------
-- MySQL specific tests and configurations.
-- $Id: mysql.lua,v 1.4 2006/01/25 20:28:30 tomas Exp $
---------------------------------------------------------------------
QUERYING_STRING_TYPE_NAME = "binary(65535)"
table.insert (CUR_METHODS, "numrows")
table.insert (EXTENSIONS, numrows)
---------------------------------------------------------------------
-- Build SQL command to create the test table.
---------------------------------------------------------------------
local _define_table = define_table
function define_table (n)
return _define_table(n) .. " TYPE = InnoDB;"
end
---------------------------------------------------------------------
-- MySQL versions 4.0.x do not implement rollback.
---------------------------------------------------------------------
local _rollback = rollback
function rollback ()
if luasql._MYSQLVERSION and string.sub(luasql._MYSQLVERSION, 1, 3) == "4.0" then
io.write("skipping rollback test (mysql version 4.0.x)")
return
else
_rollback ()
end
end

As stated in my previous comment, the error indicates that table.insert (CUR_METHODS, ...) is getting a nil as first arg. Since the first arg is CUR_METHODS, it means that this object CUR_METHODS has not been defined yet. Since this happens near top of the luasql.mysql module, my guess is that the luasql initialization was incomplete, maybe because the mysql DLL was not found. My guess is that the LUA_CPATH does not find the MySQL DLL for luasql, but I'm surprised that you wouldn't get a package error, so something odd is going on. You'll have to dig into the luasql module and C file to figure out why it is not being created.
Update: alternately, update your post to show the output of print("LUA path:", package.path) and print("LUA path:", package.cpath) from your mysql-proxy script and also show the path of folder where luasql is installed and contents of that folder.

Related

using nginx' lua to validate GitHub webhooks and delete cron-lock-file

What I have:
GNU/Linux host
nginx is up and running
there is a cron-job scheduled to run immediately after a specific file has been removed (similar to run-crons)
GitHub sends a webhook when someone pushes to a repository
What I want:
I do now want to run either lua or anything comparable to parse GitHub's request and validate it and then delete a file (if the request was valid of course).
Preferably all of this should happen without the hassle to maintain an additional PHP installation as there is currently none, or the need to use fcgiwrap or similar.
Template:
On the nginx side I have something equivalent to
location /deploy {
# execute lua (or equivalent) here
}
To read json body of GH webhook you nead use JSON4Lua lib, and to validate HMAC signature use luacrypto.
Preconfigure
Install required modules
$ sudo luarocks install JSON4Lua
$ sudo luarocks install luacrypto
In Nginx define location for deploy
location /deploy {
client_body_buffer_size 3M;
client_max_body_size 3M;
content_by_lua_file /path/to/handler.lua;
}
The max_body_size and body_buffer_size should be equal to prevent error
request body in temp file not supported
https://github.com/openresty/lua-nginx-module/issues/521
Process webhook
Get request payload data and check is correct
ngx.req.read_body()
local data = ngx.req.get_body_data()
if not data then
ngx.log(ngx.ERR, "failed to get request body")
return ngx.exit (ngx.HTTP_BAD_REQUEST)
end
Verify GH signature with use luacrypto
local function verify_signature (hub_sign, data)
local sign = 'sha1=' .. crypto.hmac.digest('sha1', data, secret)
-- this is simple comparison, but it's better to use a constant time comparison
return hub_sign == sign
end
-- validate GH signature
if not verify_signature(headers['X-Hub-Signature'], data) then
ngx.log(ngx.ERR, "wrong webhook signature")
return ngx.exit (ngx.HTTP_FORBIDDEN)
end
Parse data as json and check is master branch, for deploy
data = json.decode(data)
-- on master branch
if data['ref'] ~= branch then
ngx.say("Skip branch ", data['ref'])
return ngx.exit (ngx.HTTP_OK)
end
If all correct, call deploy function
local function deploy ()
-- run command for deploy
local handle = io.popen("cd /path/to/repo && sudo -u username git pull")
local result = handle:read("*a")
handle:close()
ngx.say (result)
return ngx.exit (ngx.HTTP_OK)
end
Example
Example constant time string compare
local function const_eq (a, b)
-- Check is string equals, constant time exec
getmetatable('').__index = function (str, i)
return string.sub(str, i, i)
end
local diff = string.len(a) == string.len(b)
for i = 1, math.min(string.len(a), string.len(b)) do
diff = (a[i] == b[i]) and diff
end
return diff
end
A complete example of how I use it in github gist https://gist.github.com/Samael500/5dbdf6d55838f841a08eb7847ad1c926
This solution does not implement verification for GitHub's hooks and assumes you have the lua extension and the cjson module installed:
location = /location {
default_type 'text/plain';
content_by_lua_block {
local cjson = require "cjson.safe"
ngx.req.read_body()
local data = ngx.req.get_body_data()
if
data
then
local obj = cjson.decode(data)
if
# checksum checking should go here
(obj and obj.repository and obj.repository.full_name) == "user/reponame"
then
local file = io.open("<your file>","w")
if
file
then
file:close()
ngx.say("success")
else
ngx.exit(ngx.HTTP_INTERNAL_SERVER_ERROR)
end
else
ngx.exit(ngx.HTTP_UNAUTHORIZED)
end
else
ngx.exit(ngx.HTTP_NOT_ALLOWED)
end
}
}

Parallel mysql I/O in Ruby

Good day to you. I'm writing a cron job that hopefully will split a huge MySQL table to several threads and do some work on them. This is the minimal sample of what I have at the moment:
require 'mysql'
require 'parallel'
#db = Mysql.real_connect("localhost", "root", "", "database")
#threads = 10
Parallel.map(1..#threads, :in_processes => 8) do |i|
begin
#db.query("SELECT url FROM pages LIMIT 1 OFFSET #{i}")
rescue Mysql::Error => e
#db.reconnect()
puts "Error code: #{e.errno}"
puts "Error message: #{e.error}"
puts "Error SQLSTATE: #{e.sqlstate}" if e.respond_to?("sqlstate")
end
end
#db.close
The threads don't need to return anything, they get their job share and they do it. Only they don't. Either connection to MySQL is lost during the query, or connection doesn't exist (MySQL server has gone away?!), or no _dump_data is defined for class Mysql::Result and then Parallel::DeadWorker.
How to do that right?
map method expects a result; I don't need a result, so I switched to each:
Parallel.each(1..#threads, :in_processes => 8) do |i|
Also this solves a problem with MySQL: I just needed to start the connection inside the parallel process. When using each loop, it's possible. Of course, connection should be closed inside the process also.

Bind9 and MySQL DLZ Buffer Error

I compiled Bind 9 from source (see below) and set up Bind9 with MySQL DLZ.
I keep getting an error when I attempt to fetch anything from the server about buffer overflow. I've googled many times but can not find anything on how to fix this error.
Configure options:
root#anacrusis:/opt/bind9/bind-9.9.1-P3# named -V BIND 9.9.1-P3 built
with '--prefix=/opt/bind9' '--mandir=/opt/bind9/man'
'--infodir=/opt/bind9/info' '--sysconfdir=/opt/bind9/config'
'--localstatedir=/opt/bind9/var' '--enable-threads'
'--enable-largefile' '--with-libtool' '--enable-shared'
'--enable-static' '--with-openssl=/usr' '--with-gssapi=/usr'
'--with-gnu-ld' '--with-dlz-postgres=no' '--with-dlz-mysql=yes'
'--with-dlz-bdb=no' '--with-dlz-filesystem=yes' '--with-dlz-stub=yes'
'--with-dlz-ldap=yes' '--enable-ipv6' 'CFLAGS=-fno-strict-aliasing
-DDIG_SIGCHASE -O2' 'LDFLAGS=-Wl,-Bsymbolic-functions' 'CPPFLAGS=' using OpenSSL version: OpenSSL 1.0.1 14 Mar 2012 using libxml2
version: 2.7.8
This is the error I get when I dig example.com (with debug):
Query String: select ttl, type, mx_priority, case when
lower(type)='txt' then concat('"', data, '"')
else data end from dns_records where zone = 'example.com' and host = '#'
17-Sep-2012 01:09:33.610 dns_rdata_fromtext: buffer-0x7f5bfca73360:1:
unexpected end of input 17-Sep-2012 01:09:33.610 dns_sdlz_putrr
returned error. Error code was: unexpected end of input 17-Sep-2012
01:09:33.610 Query String: select ttl, type, mx_priority, case when
lower(type)='txt' then concat('"', data, '"')
else data end from dns_records where zone = 'example.com' and host = '*'
17-Sep-2012 01:09:33.610 query.c:2579: fatal error: 17-Sep-2012
01:09:33.610 RUNTIME_CHECK(result == 0) failed 17-Sep-2012
01:09:33.610 exiting (due to fatal error in library)
Named.conf
options {
directory "/opt/bind9/";
allow-query-cache { none; };
allow-query { any; };
recursion no;
};
dlz "Mysql zone" {
database "mysql
{host=localhost dbname=system ssl=false user=root pass=*password*}
{select zone from dns_records where zone = '$zone$'}
{select ttl, type, mx_priority, case when lower(type)='txt' then concat('\"', data, '\"')
else data end from dns_records where zone = '$zone$' and host = '$record$'}
{}
{}
{}
{}";
};
Do you run named single-threaded (with "-n 1" parameter)? If not, named will crash in various places when working on more than one query in parallel, since the MySQL DLZ module is not thread safe.
Manually log into the DB and run the query. See what it comes up with. The error says it's got an unexpected end of input, meaning it was expecting to get something and it never got it. So the first thing is to see if it you can get it manually. Maybe the mysqld isn't running. Maybe the user isn't defined or password is set wrong or permissions are not granted on that table. These could all account for the error.
Assuming all this works then you have two options: Enable more logging in your named.conf so you have more data to work with on what's happeningRemove and reinstall BIND, ensuring that all hashes match on all libraries and that all dependancies are in place.
I have gotten Bind with DLZ working on CentOS 7. I do not get the error that is effecting you.
I realize this is an older post but I thought I would share my conf files , and configure options.
I am using Bind 9.11.0
configure
./configure --prefix=/usr --sysconfdir=/etc/bind --localstatedir=/var --mandir=/usr/share/man --infodir=/usr/share/info --enable-threads --enable-largefile --with-libtool --enable-shared --enable-static --with-openssl=/usr --with-gssapi=/usr --with-gnu-ld --with-dlz-postgres=no --with-dlz-mysql=yes --with-dlz-bdb=no --with-dlz-filesystem=yes --with-dlz-stub=yes --enable-ipv6
named.conf
// This is the primary configuration file for the BIND DNS server named.
//
// Please read /usr/share/doc/bind9/README.Debian.gz for information on the
// structure of BIND configuration files in Debian, *BEFORE* you customize
// this configuration file.
//
// If you are just adding zones, please do that in /etc/bind/named.conf.local
#auskommentiert !!!
#include "/etc/bind/named.conf.options";
#include "/etc/bind/named.conf.local";
#include "/etc/bind/named.conf.default-zones";
key "rndc-key" {
// how was key encoded
algorithm hmac-md5;
// what is the pass-phrase for the key
secret "noway";
};
#options {
#default-key "rndc-key";
#default-server 127.0.0.1;
#default-port 953;
#};
controls {
inet * port 953 allow { any; } keys { "rndc-key"; };
#inet * port 53 allow { any; } keys { "rndc-key"; };
};
logging {
channel query.log {
file "/var/log/query.log";
// Set the severity to dynamic to see all the debug messages.
severity dynamic;
};
category queries { query.log; };
};
dlz "Mysql zone" {
database "mysql
{host=172.16.254.100 port=3306 dbname=dyn_server_db user=db_user pass=db_password}
{SELECT zone FROM dyn_dns_records WHERE zone = '$zone$'}
{SELECT ttl, type, mx_priority, IF(type = 'TXT', CONCAT('\"',data,'\"'), data) AS data
FROM dyn_dns_records
WHERE zone = '$zone$' AND host = '$record$' AND type <> 'SOA' AND type <> 'NS'}
{SELECT ttl, type, data, primary_ns, resp_person, serial, refresh, retry, expire, minimum
FROM dyn_dns_records
WHERE zone = '$zone$' AND (type = 'SOA' OR type='NS')}
{SELECT ttl, type, host, mx_priority, IF(type = 'TXT', CONCAT('\"',data,'\"'), data) AS data, resp_person, serial, refresh, retry, expire, minimum
FROM dyn_dns_records
WHERE zone = '$zone$' AND type <> 'SOA' AND type <> 'NS'}
{SELECT zone FROM xfr_table where zone='$zone$' AND client = '$client$'}";
};

How to trace MySql queries using MySql-Proxy?

I just downloaded the mysql-proxy and created this script lua (found in Mysql docs):
function read_query(packet)
if string.byte(packet) == proxy.COM_QUERY then
print("QUERY: " .. string.sub(packet, 2))
end
end
This is the command-line I'm using:
mysql-proxy -P localhost:1234 -b localhost:3306 --proxy-lua-script=profile.lua --plugins=proxy
When I run a simple query (like "select * from table1"), this error is reported: "failed: .\lua-scope.c:241: stat(C:...\profile.lua) failed: No error (0)"
Note: If I run mysql-proxy without lua script, no error occurs.
I need to install something to get mysql-proxy and query tracing working?
My environment is Windows 7 Professional x64.
Sorry the bad english.
The error you're getting is caused by --proxy-lua-script pointing to a file that mysql-proxy can't find. Either you've typed the name in wrong, you've typed the path in wrong, or you are expecting it in your CWD and it's not there. Or actually, looking at the entire error a little more closely, it seems possible that mysql-proxy itself sees the file in CWD itself OK, but one of the underlying modules doesn't like it (possibly because mysql-proxy changes the CWD somehow?)
Try saving profile.lua to the root of your C: drive and trying different versions of the option like so:
--proxy-lua-script=c:\profile.lua
--proxy-lua-script=\profile.lua
--proxy-lua-script=/profile.lua
One of those would probably work
simple query log lua script:
require("mysql.tokenizer")
local fh = io.open("/var/log/mysql/proxy.query.log", "a+")
fh:setvbuf('line',4096)
local the_query = "";
local seqno = 0;
function read_query( packet )
if string.byte(packet) == proxy.COM_QUERY then
seqno = seqno + 1
the_query = (string.gsub(string.gsub(string.sub(packet, 2), "%s%s*", ' '), "^%s*(.-)%s*$", "%1"))
fh:write(string.format("%s %09d %09d : %s (%s) -- %s\n",
os.date('%Y-%m-%d %H:%M:%S'),
proxy.connection.server.thread_id,
seqno,
proxy.connection.client.username,
proxy.connection.client.default_db,
the_query))
fh:flush()
return proxy.PROXY_SEND_QUERY
else
query = ""
end
end

How to connect & query MySQL from within Lua?

How can I connect to a MySQL database from using Lua programming language?
If a good/popular library exists, what is it?
Minimal woking example for LuaSQL - simple interface from Lua to a DBMS.
package.cpath = package.cpath .. ";/usr/lib/i386-linux-gnu/lua/5.1/?.so"
luasql = require "luasql.mysql"
env = assert (luasql.mysql())
con = assert (env:connect("dbname","user","password"))
cur = assert (con:execute("SHOW TABLES"))
row = cur:fetch ({}, "a")
while row do
print(string.format("Name: %s", row.Tables_in_dbname))
row = cur:fetch (row, "a")
end
Line 1 used if module luasql.mysql not found. Also environment variable LUA_CPATH may be used.
In case your mysql database is remote, you can add host as another optional parameter to connect. Port can follow host as well:
con = assert (env:connect("dbname","user","password","host",port))
From LuaSQL -- Database connectivity for the Lua programming language:
require "luasql.mysql"
env = assert (luasql.mysql())
con = assert (env:connect"my_db")
for id, name, address in rows (con, "select * from contacts") do
print (string.format ("%s: %s", name, address))
end