I am building a C application on Linux platform. I need to use libmysqlclient for interfacing to database.
I downloaded Linux source code package mysql-connector-c-6.0.2.tar.gz. I compiled it as per the instructions. I get the below libraries:
libmysqlclient.a libmysqlclient.so libmysql.so.16
libmysqlclient_r.so libmysql.so libmysql.so.16.0.0
If my application is multi-threaded, can I link my application with libmysqlclient.a? As per mysql documentation (http://forge.mysql.com/wiki/Autotools_to_CMake_Transition_Guide), with cmake tool, clients are always thread safe.
After linking my application with libmysqlclient.a, I get a crash in my application with below call stack:
#0 0x0867878a in my_stat ()
No symbol table info available.
#1 0x08671611 in init_available_charsets.clone.0 ()
No symbol table info available.
#2 0x086720d5 in get_charset_by_csname ()
No symbol table info available.
#3 0x086522af in mysql_init_character_set ()
No symbol table info available.
#4 0x0865266d in mysql_real_connect ()
In my application, I have below code in the thread function:
if (NULL == (pMySQL = mysql_init(NULL)))
{
return -1;
}
if (NULL == mysql_real_connect(pMySQL, ServerName, UserName, Password, Name, Port, NULL, 0))
{
mysql_close(pMySQL);
return -1;
}
if (0 != mysql_query(pMySQL, pQuery))
{
mysql_close(pMySQL);
return -1;
}
mysql_close(pMySQL);
I am not using libmysqlclient_r.so as I want to link to mysql client library statically. Is there any way to generate libmysqlclient_r.a with cmake?
Update:
Without doing anything else, I just changed mysql client build type to debug. Now I get the crash in mysql_init() function.
On the application console, I get below print:
safe_mutex: Trying to lock unitialized mutex at /install/mysqlconnc/mysql-connector-c-6.0.2/mysys/safemalloc.c, line 520
The call stack of the crash is as below:
#0 0x00556430 in __kernel_vsyscall ()
No symbol table info available.
#1 0x45fdf2f1 in raise () from /lib/libc.so.6
No symbol table info available.
#2 0x45fe0d5e in abort () from /lib/libc.so.6
No symbol table info available.
#3 0x086833e5 in safe_mutex_lock (mp=0x915e8e0, my_flags=0,
file=0x895b9d8 "/install/mysqlconnc/mysql-connector-c-6.0.2/mysys/safemalloc.c", line=520)
at /install/mysqlconnc/mysql-connector-c-6.0.2/mysys/thr_mutex.c:178
error = 140915306
__PRETTY_FUNCTION__ = "safe_mutex_lock"
#4 0x08682715 in _sanity (
filename=0x895a87c "/install/mysqlconnc/mysql-connector-c-6.0.2/mysys/my_error.c", lineno=195)
at /install/mysqlconnc/mysql-connector-c-6.0.2/mysys/safemalloc.c:520
irem = 0xf2300468
flag = 0
count = 0
#5 0x0868186b in _mymalloc (size=16,
filename=0x895a87c "/install/mysqlconnc/mysql-connector-c-6.0.2/mysys/my_error.c", lineno=195, MyFlags=16)
at /install/mysqlconnc/mysql-connector-c-6.0.2/mysys/safemalloc.c:130
irem = 0x0
data = 0x0
_db_stack_frame_ = {func = 0x6d617266 <Address 0x6d617266 out of bounds>, file = 0x65685f65 <Address 0x65685f65 out of bounds>,
level = 0, prev = 0x0}
#6 0x0867e0e1 in my_error_register (errmsgs=0x89a7760, first=2000, last=2058)
at /install/mysqlconnc/mysql-connector-c-6.0.2/mysys/my_error.c:194
meh_p = 0x46087568
search_meh_pp = 0x1000
#7 0x08655f7e in init_client_errs ()
at /install/mysqlconnc/mysql-connector-c-6.0.2/libmysql/errmsg.c:238
No locals.
#8 0x08655fe3 in mysql_server_init (argc=0, argv=0x0, groups=0x0)
at /install/mysqlconnc/mysql-connector-c-6.0.2/libmysql/libmysql.c:128
result = 0
#9 0x08651fc0 in mysql_init (mysql=0x0)
at /install/mysqlconnc/mysql-connector-c-6.0.2/libmysql/client.c:1606
Solution:
I put a call to mysql_library_init() before creation of threads and put a call to mysql_library_end() after termination of threads. In each thread, I put a call to mysql_thread_init() at the start of thread function and put a call to mysql_thread_end() at the end of thread function. This solved the problem of crashing.
Update:
It seems that you need to call mysql_library_init() before mysql_init():
You must either call mysql_library_init()
prior to spawning any threads, or else use a mutex to protect the
call, whether you invoke mysql_library_init() or indirectly through
mysql_init(). Do this prior to any other client library call.
Regarding your original question, libmysqlclient_r.so is actually a symbolic link to libmysql.so. You can change libmysql/CMakeLists.txt to produce a static library (libmysql.a) instead by removing the SHARED keyword from the following line:
ADD_LIBRARY(libmysql SHARED ${CLIENT_SOURCES} libmysql.def)
However, I would recommend (1) trying to run the same code without using threads and see if the problem persists, (2) building and using the debug version of the libraries:
cmake -G "Unix Makefiles" -DCMAKE_BUILD_TYPE=Debug
make
This way you could investigate the problem in more details.
Related
I am trying to write a simple Ada (with AWS) program to post data to a server. The curl command is working as follows and return a valid response in JSON after successful login:
curl -XPOST -d '{"type":"m.login.password", "user":"xxx", "password": "xxxxxxxxxx"}' "https://matrix.org/_matrix/client/r0/login"
My Ada program:
with Ada.Exceptions; use Ada.Exceptions;
with Ada.Text_Io; use Ada.Text_IO;
with AWS.Client;
with AWS.Communication.Client;
with AWS.MIME;
with AWS.Net;
with AWS.Response;
use AWS;
procedure Communicate is
Result : Response.Data;
Data : String := "{""type"":""m.login.password"", ""user"":""xxx"", ""password"": ""xxxxxxxxxx""}";
begin
Result := Client.Post
( URL => "https://matrix.org/_matrix/client/r0/login",
Data => Data,
Content_Type => AWS.MIME.Application_JSON ) ;
Put_Line ( Response.Message_Body ( Result ) ) ;
end Communicate;
An exception was raised. I can't figure out what is wrong with this code.
$ ./Communicate
raised PROGRAM_ERROR : aws-client.adb:543 finalize/adjust raised exception
To test the code, you can create an account at http://matrix.org and replace the login credential.
Thanks.
Adrian
After a few minor changes (mostly because I don't like compiler warnings), and an adaption to the Debian/Jessie version of AWS, I got it to work.
Here's the adapted version:
with Ada.Text_IO; use Ada.Text_IO;
with AWS.Client;
-- with AWS.MIME;
with AWS.Response;
use AWS;
procedure Communicate is
Result : Response.Data;
Data : constant String :=
"{""type"":""m.login.password"", ""user"":""xxx"", " &
"""password"": ""xxxxxxxxxx""}";
begin
Result := Client.Post
(URL => "https://matrix.org/_matrix/client/r0/login",
Data => Data,
Content_Type => "application/json");
-- Content_Type => AWS.MIME.Application_JSON);
Put_Line (Response.Message_Body (Result));
end Communicate;
Here is my project file:
with "aws";
project Communicate is
for Main use ("communicate");
package Builder is
for Default_Switches ("Ada")
use ("-m");
end Builder;
package Compiler is
for Default_Switches ("Ada")
use ("-fstack-check", -- Generate stack checking code (part of Ada)
"-gnata", -- Enable assertions (part of Ada)
"-gnato13", -- Overflow checking (part of Ada)
"-gnatf", -- Full, verbose error messages
"-gnatwa", -- All optional warnings
"-gnatVa", -- All validity checks
"-gnaty3abcdefhiklmnoOprstux", -- Style checks
"-gnatwe", -- Treat warnings as errors
"-gnat2012", -- Use Ada 2012
"-Wall", -- All GCC warnings
"-O2"); -- Optimise (level 2/3)
end Compiler;
end Communicate;
I built the program with:
% gprbuild -P communicate
gnatgcc -c -fstack-check -gnata -gnato13 -gnatf -gnatwa -gnatVa -gnaty3abcdefhiklmnoOprstux -gnatwe -gnat2012 -Wall -O2 communicate.adb
gprbind communicate.bexch
gnatbind communicate.ali
gnatgcc -c b__communicate.adb
gnatgcc communicate.o -L/usr/lib/x86_64-linux-gnu -lgnutls -lz -llber -lldap -lpthread -o communicate
%
And then tested with:
% ./communicate
{"errcode":"M_FORBIDDEN","error":"Invalid password"}
%
It looks like the problem is located in your AWS version/installation.
Problem resolved by building AWS with gnutls from MacPorts. Apple deprecated OpenSSL since OS X Lion and used CommonCrypto so modern macOS does not come with OpenSSL. The solution is to download and install OpenSSL or gnutls from Mac Ports or Home Brew.
Another problem is that Apple introduced SIP (System Integrity Protection) since El Capitan. With SIP enabled, user with administrator's rights is unable to change the contents in /usr/include and /usr/lib etc.
Mac Ports installs to /opt/local so I made references to /opt/local/include and /opt/local/lib so that AWS can build with either OpenSSL or gnutls.
i am trying to compile my small project (a yesod application with lambdacms) on nixos. However, after using cabal2nix (more precisely cabal2nix project-karma.cabal --sha256=0 --shell > shell.nix) , I am still missing a dependency wrt. postgresql it seems.
My shell.nix file looks like this:
{ nixpkgs ? import <nixpkgs> {}, compiler ? "default" }:
let
inherit (nixpkgs) pkgs;
f = { mkDerivation, aeson, base, bytestring, classy-prelude
, classy-prelude-conduit, classy-prelude-yesod, conduit, containers
, data-default, directory, fast-logger, file-embed, filepath
, hjsmin, hspec, http-conduit, lambdacms-core, monad-control
, monad-logger, persistent, persistent-postgresql
, persistent-template, random, resourcet, safe, shakespeare, stdenv
, template-haskell, text, time, transformers, unordered-containers
, uuid, vector, wai, wai-extra, wai-logger, warp, yaml, yesod
, yesod-auth, yesod-core, yesod-form, yesod-static, yesod-test
}:
mkDerivation {
pname = "karma";
version = "0.0.0";
sha256 = "0";
isLibrary = true;
isExecutable = true;
libraryHaskellDepends = [
aeson base bytestring classy-prelude classy-prelude-conduit
classy-prelude-yesod conduit containers data-default directory
fast-logger file-embed filepath hjsmin http-conduit lambdacms- core
monad-control monad-logger persistent persistent-postgresql
persistent-template random safe shakespeare template-haskell text
time unordered-containers uuid vector wai wai-extra wai-logger warp
yaml yesod yesod-auth yesod-core yesod-form yesod-static
nixpkgs.zlib
nixpkgs.postgresql
nixpkgs.libpqxx
];
libraryPkgconfigDepends = [ persistent-postgresql];
executableHaskellDepends = [ base ];
testHaskellDepends = [
base classy-prelude classy-prelude-yesod hspec monad-logger
persistent persistent-postgresql resourcet shakespeare transformers
yesod yesod-core yesod-test
];
license = stdenv.lib.licenses.bsd3;
};
haskellPackages = if compiler == "default"
then pkgs.haskellPackages
else pkgs.haskell.packages.${compiler};
drv = haskellPackages.callPackage f {};
in
if pkgs.lib.inNixShell then drv.env else drv
The output is as follows:
markus#nixos ~/git/haskell/karma/karma (git)-[master] % nix-shell --command `stack build`
postgresql-libpq-0.9.1.1: configure
ReadArgs-1.2.2: download
postgresql-libpq-0.9.1.1: build
ReadArgs-1.2.2: configure
ReadArgs-1.2.2: build
ReadArgs-1.2.2: install
-- While building package postgresql-libpq-0.9.1.1 using:
/run/user/1000/stack31042/postgresql-libpq-0.9.1.1/.stack-work/dist/x86_64-linux/Cabal-1.22.4.0/setup/setup --builddir=.stack-work/dist/x86_64-linux/Cabal-1.22.4.0/ build --ghc-options " -ddump-hi -ddump-to-file"
Process exited with code: ExitFailure 1
Logs have been written to: /home/markus/git/haskell/karma/karma/.stack-work/logs/postgresql-libpq-0.9.1.1.log
[1 of 1] Compiling Main ( /run/user/1000/stack31042/postgresql-libpq-0.9.1.1/Setup.hs, /run/user/1000/stack31042/postgresql-libpq-0.9.1.1/.stack-work/dist/x86_64-linux/Cabal-1.22.4.0/setup/Main.o )
Linking /run/user/1000/stack31042/postgresql-libpq-0.9.1.1/.stack-work/dist/x86_64-linux/Cabal-1.22.4.0/setup/setup ...
Configuring postgresql-libpq-0.9.1.1...
Building postgresql-libpq-0.9.1.1...
Preprocessing library postgresql-libpq-0.9.1.1...
LibPQ.hsc:213:22: fatal error: libpq-fe.h: No such file or directory
compilation terminated.
compiling .stack-work/dist/x86_64-linux/Cabal-1.22.4.0/build/Database/PostgreSQL/LibPQ_hsc_make.c failed (exit code 1)
command was: /nix/store/9fbfiij3ajnd3fs1zyc2qy0ispbszrr7-gcc-wrapper-4.9.3/bin/gcc -c .stack-work/dist/x86_64-linux/Cabal-1.22.4.0/build/Database/PostgreSQL/LibPQ_hsc_make.c -o .stack-work/dist/x86_64-linux/Cabal-1.22.4.0/build/Database/PostgreSQL/LibPQ_hsc_make.o -fno-stack-protector -D__GLASGOW_HASKELL__=710 -Dlinux_BUILD_OS=1 -Dx86_64_BUILD_ARCH=1 -Dlinux_HOST_OS=1 -Dx86_64_HOST_ARCH=1 -I/run/current-system/sw/include -Icbits -I.stack-work/dist/x86_64-linux/Cabal-1.22.4.0/build/autogen -include .stack-work/dist/x86_64-linux/Cabal-1.22.4.0/build/autogen/cabal_macros.h -I/nix/store/xphvly2zcd6jsc2xklz1zmmz4y0dh3ny-ghc-7.10.2/lib/ghc-7.10.2/bytes_6elQVSg5cWdFrvRnfxTUrH/include -I/nix/store/xphvly2zcd6jsc2xklz1zmmz4y0dh3ny-ghc-7.10.2/lib/ghc-7.10.2/base_GDytRqRVSUX7zckgKqJjgw/include -I/nix/store/6ykqcjxr74l642kv9gf1ib8v9yjsgxr9-gmp-5.1.3/include -I/nix/store/xphvly2zcd6jsc2xklz1zmmz4y0dh3ny-ghc-7.10.2/lib/ghc-7.10.2/integ_2aU3IZNMF9a7mQ0OzsZ0dS/include -I/nix/store/xphvly2zcd6jsc2xklz1zmmz4y0dh3ny-ghc-7.10.2/lib/ghc-7.10.2/include -I/nix/store/xphvly2zcd6jsc2xklz1zmmz4y0dh3ny-ghc-7.10.2/lib/ghc-7.10.2/include/
I assume not much is missing, so a pointer would be nice.
What is also weird, that is that "nix-shell" works but following that up with "stack exec yesod devel" tells me
Resolving dependencies...
Configuring karma-0.0.0...
cabal: At least the following dependencies are missing:
classy-prelude >=0.10.2,
classy-prelude-conduit >=0.10.2,
classy-prelude-yesod >=0.10.2,
hjsmin ==0.1.*,
http-conduit ==2.1.*,
lambdacms-core >=0.3.0.2 && <0.4,
monad-logger ==0.3.*,
persistent >=2.0 && <2.3,
persistent-postgresql >=2.1.1 && <2.3,
persistent-template >=2.0 && <2.3,
uuid >=1.3,
wai-extra ==3.0.*,
warp >=3.0 && <3.2,
yesod >=1.4.1 && <1.5,
yesod-auth >=1.4.0 && <1.5,
yesod-core >=1.4.6 && <1.5,
yesod-form >=1.4.0 && <1.5,
yesod-static >=1.4.0.3 && <1.6
When using mysql instead, I am getting
pcre-light-0.4.0.4: configure
mysql-0.1.1.8: configure
mysql-0.1.1.8: build
Progress: 2/59
-- While building package mysql-0.1.1.8 using:
/run/user/1000/stack12820/mysql-0.1.1.8/.stack-work/dist/x86_64-linux/Cabal-1.22.4.0/setup/setup --builddir=.stack-work/dist/x86_64- linux/Cabal-1.22.4.0/ build --ghc-options " -ddump-hi -ddump-to-file"
Process exited with code: ExitFailure 1
Logs have been written to: /home/markus/git/haskell/karma/karma/.stack-work/logs/mysql-0.1.1.8.log
[1 of 1] Compiling Main ( /run/user/1000/stack12820/mysql-0.1.1.8/Setup.lhs, /run/user/1000/stack12820/mysql-0.1.1.8/.stack-work/dist/x86_64-linux/Cabal-1.22.4.0/setup/Main.o )
Linking /run/user/1000/stack12820/mysql-0.1.1.8/.stack-work/dist/x86_64-linux/Cabal-1.22.4.0/setup/setup ...
Configuring mysql-0.1.1.8...
Building mysql-0.1.1.8...
Preprocessing library mysql-0.1.1.8...
In file included from C.hsc:68:0:
include/mysql_signals.h:9:19: fatal error: mysql.h: No such file or directory
#include "mysql.h"
^
compilation terminated.
compiling .stack-work/dist/x86_64-linux/Cabal-1.22.4.0/build/Database/MySQL/Base/C_hsc_make.c failed (exit code 1)
command was: /nix/store/9fbfiij3ajnd3fs1zyc2qy0ispbszrr7-gcc-wrapper-4.9.3/bin/gcc -c .stack-work/dist/x86_64-linux/Cabal-1.22.4.0/build/Database/MySQL/Base/C_hsc_make.c -o .stack-work/dist/x86_64-linux/Cabal-1.22.4.0/build/Database/MySQL/Base/C_hsc_make.o -fno-stack-protector -D__GLASGOW_HASKELL__=710 -Dlinux_BUILD_OS=1 -Dx86_64_BUILD_ARCH=1 -Dlinux_HOST_OS=1 -Dx86_64_HOST_ARCH=1 -I/nix/store/7ppa4k2drrvjk94rb60c1df9nvw0z696-mariadb-10.0.22-lib/include -I/nix/store/7ppa4k2drrvjk94rb60c1df9nvw0z696-mariadb-10.0.22-lib/include/.. -Iinclude -I.stack-work/dist/x86_64-linux/Cabal-1.22.4.0/build/autogen -include .stack-work/dist/x86_64-linux/Cabal-1.22.4.0/build/autogen/cabal_macros.h -I/nix/store/xphvly2zcd6jsc2xklz1zmmz4y0dh3ny-ghc-7.10.2/lib/ghc-7.10.2/bytes_6elQVSg5cWdFrvRnfxTUrH/include -I/nix/store/xphvly2zcd6jsc2xklz1zmmz4y0dh3ny-ghc-7.10.2/lib/ghc-7.10.2/base_GDytRqRVSUX7zckgKqJjgw/include -I/nix/store/6ykqcjxr74l642kv9gf1ib8v9yjsgxr9-gmp-5.1.3/include -I/nix/store/xphvly2zcd6jsc2xklz1zmmz4y0dh3ny-ghc-7.10.2/lib/ghc-7.10.2/integ_2aU3IZNMF9a7mQ0OzsZ0dS/include -I/nix/store/xphvly2zcd6jsc2xklz1zmmz4y0dh3ny-ghc-7.10.2/lib/ghc-7.10.2/include -I/nix/store/xphvly2zcd6jsc2xklz1zmmz4y0dh3ny-ghc-7.10.2/lib/ghc-7.10.2/include/
-- While building package pcre-light-0.4.0.4 using:
/home/markus/.stack/setup-exe-cache/setup-Simple-Cabal-1.22.4.0-x86_64-linux-ghc-7.10.2 --builddir=.stack-work/dist/x86_64-linux/Cabal-1.22.4.0/ configure --with-ghc=/run/current-system/sw/bin/ghc --user --package-db=clear --package-db=global --package-db=/home/markus/.stack/snapshots/x86_64-linux/nightly-2015-11-17/7.10.2/pkgdb/ --libdir=/home/markus/.stack/snapshots/x86_64-linux/nightly-2015-11-17/7.10.2/lib --bindir=/home/markus/.stack/snapshots/x86_64-linux/nightly-2015-11-17/7.10.2/bin --datadir=/home/markus/.stack/snapshots/x86_64-linux/nightly-2015-11-17/7.10.2/share --libexecdir=/home/markus/.stack/snapshots/x86_64-linux/nightly-2015-11-17/7.10.2/libexec --sysconfdir=/home/markus/.stack/snapshots/x86_64-linux/nightly-2015-11-17/7.10.2/etc --docdir=/home/markus/.stack/snapshots/x86_64-linux/nightly-2015-11-17/7.10.2/doc/pcre-light-0.4.0.4 --htmldir=/home/markus/.stack/snapshots/x86_64-linux/nightly-2015-11-17/7.10.2/doc/pcre-light-0.4.0.4 --haddockdir=/home/markus/.stack/snapshots/x86_64-linux/nightly-2015-11-17/7.10.2/doc/pcre-light-0.4.0.4 --dependency=base=base-4.8.1.0-4f7206fd964c629946bb89db72c80011 --dependency=bytestring=bytestring-0.10.6.0-18c05887c1aaac7adb3350f6a4c6c8ed
Process exited with code: ExitFailure 1
Logs have been written to: /home/markus/git/haskell/karma/karma/.stack-work/logs/pcre-light-0.4.0.4.log
Configuring pcre-light-0.4.0.4...
setup-Simple-Cabal-1.22.4.0-x86_64-linux-ghc-7.10.2: The program 'pkg-config'
version >=0.9.0 is required but it could not be found.
After adding pkgconfig to my global configuration, the build seems to get a little further ahead, so it seems that shell.nix is ignored somewhat.
(Sources for what I tried so far:
https://groups.google.com/forum/#!topic/haskell-stack/_ZBh01VP_fo)
Update: It seems like I overlooked this section of the manual
http://nixos.org/nixpkgs/manual/#using-stack-together-with-nix
However, the first idea that came to mind
(stack --extra-lib-dirs=/nix/store/c6qy7n5wdwl164lnzha7vpc3av9yhnga-postgresql-libpq-0.9.1.1/lib build)
did not work yet, most likely I need to use
--extra-include-dirs or try one of the variations. It seems weird that stack is still trying to build postgresql-libpq in the very same version, though.
Update2: Currently trying out "stack --extra-lib-dirs=/nix/store/1xf77x47d0m23nbda0azvkvj8w8y77c7-postgresql-9.4.5/lib --extra-include-dirs=/nix/store/1xf77x47d0m23nbda0azvkvj8w8y77c7-postgresql-9.4.5/include build" which looks promising. Does not look like the nix-way, but still.
Update3: Still getting
<command line>: can't load .so/.DLL for: /home/markus /.stack/snapshots/x86_64-linux/nightly-2015-11-17/7.10.2/lib/x86_64-linux- ghc-7.10.2/postgresql-libpq-0.9.1.1-ABGs5p1J8FbEwi6uvHaiV6/libHSpostgresql-libpq-0.9.1.1-ABGs5p1J8FbEwi6uvHaiV6-ghc7.10.2.so
(libpq.so.5: cannot open shared object file: No such file or directory) stack build 186.99s user 2.93s system 109% cpu 2:52.76 total
which is strange since libpq.so.5 is contained in /nix/store/1xf77x47d0m23nbda0azvkvj8w8y77c7-postgresql-9.4.5/lib.
An additional
$LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:/nix/store/1xf77x47d0m23nbda0azvkvj8w8y77c7-postgresql-9.4.5/lib
does not help either.
Update4:
By the way, yesod devel does the same as stack exec yesod devel. My libraries are downloaded to /nix/store but they are not recognized.
Maybe I need to make "build-nix" work and yesod devel does not work here?
Just for completeness, here is stack.yaml
resolver: nightly-2015-11-17
#run stack setup otherwise!!
# Local packages, usually specified by relative directory name
packages:
- '.'
# Packages to be pulled from upstream that are not in the resolver (e.g., acme-missiles-0.3)
extra-deps: [lambdacms-core-0.3.0.2 , friendly-time-0.4, lists-0.4.2, list-extras-0.4.1.4 ]
# Override default flag values for local packages and extra-deps
flags:
karma:
library-only: false
dev: false
# Extra package databases containing global packages
extra-package-dbs: []
Next weekend, I will check out
https://pr06lefs.wordpress.com/2014/09/27/compiling-a-yesod-project-on-nixos/
and other search results.
Funny, because I've just had a similar problem myself - solved it by adding these two lines to stack.yaml:
extra-include-dirs: [/nix/store/jrdvjvf0w9nclw7b4k0pdfkljw78ijgk-postgresql-9.4.5/include/]
extra-lib-dirs: [/nix/store/jrdvjvf0w9nclw7b4k0pdfkljw78ijgk-postgresql-9.4.5/lib/]
You may want to check first which postgresql's path from the /nix/store you should use with include/ and lib/:
nix-build --no-out-link "<nixpkgs>" -A postgresql
And BTW, why do you use nix-shell if you are going to use stack and you have project-karma.cabal available..? Have you considered migrating your project with stack init..?
Looks like stack is trying to build haskellPackages.postgresql-libpq outside of the nix framework.
You probably don't want that to happen. Maybe try to add postgresql-libpq to libraryHaskellDepends?
I am trying to connect to a mysql server using LuaSql via a mysql proxy. I try to execute a simple program (db.lua):
require("luasql.mysql")
local _sqlEnv = assert(luasql.mysql())
local _con = nil
function read_auth(auth)
local host, port = string.match(proxy.backends[1].address, "(.*):(.*)")
_con = assert(_sqlEnv:connect( "db_name", "username", "password", "hostname", "3306"))
end
function disconnect_client()
assert(_con:close())
end
function read_query(packet)
local cur = con:execute("select * from t1")
myTable = {}
row = cur:fetch(myTable, "a")
print(myTable.id,myTable.user)
end
This code executes well when I execute it without mysql-proxy. When I am connecting with mysql-proxy, the error-log displays these errors:
mysql.lua:8: bad argument #1 to 'insert' (table expected, got nil)
db.lua:1: loop or previous error loading module 'luasql.mysql'
mysql.lua is a default file of LuaSql:
---------------------------------------------------------------------
-- MySQL specific tests and configurations.
-- $Id: mysql.lua,v 1.4 2006/01/25 20:28:30 tomas Exp $
---------------------------------------------------------------------
QUERYING_STRING_TYPE_NAME = "binary(65535)"
table.insert (CUR_METHODS, "numrows")
table.insert (EXTENSIONS, numrows)
---------------------------------------------------------------------
-- Build SQL command to create the test table.
---------------------------------------------------------------------
local _define_table = define_table
function define_table (n)
return _define_table(n) .. " TYPE = InnoDB;"
end
---------------------------------------------------------------------
-- MySQL versions 4.0.x do not implement rollback.
---------------------------------------------------------------------
local _rollback = rollback
function rollback ()
if luasql._MYSQLVERSION and string.sub(luasql._MYSQLVERSION, 1, 3) == "4.0" then
io.write("skipping rollback test (mysql version 4.0.x)")
return
else
_rollback ()
end
end
As stated in my previous comment, the error indicates that table.insert (CUR_METHODS, ...) is getting a nil as first arg. Since the first arg is CUR_METHODS, it means that this object CUR_METHODS has not been defined yet. Since this happens near top of the luasql.mysql module, my guess is that the luasql initialization was incomplete, maybe because the mysql DLL was not found. My guess is that the LUA_CPATH does not find the MySQL DLL for luasql, but I'm surprised that you wouldn't get a package error, so something odd is going on. You'll have to dig into the luasql module and C file to figure out why it is not being created.
Update: alternately, update your post to show the output of print("LUA path:", package.path) and print("LUA path:", package.cpath) from your mysql-proxy script and also show the path of folder where luasql is installed and contents of that folder.
I want to know if there's a way to run some code on the child process when the parent process tries to terminate the child process. Is there a way we can write an Exception maybe?
My code looks something like this:
main_process.py
import Process from multiprocessing
def main():
p1 = Process(target = child, args = (arg1, ))
p1.start()
p1.daemon = True
#blah blah blah code here
sleep(5)
p1.terminate()
def child(arg1):
#blah blah blah
itemToSend = {}
#more blah blah
snmpEngine.transportDispatcher.jobStarted(1) # this job would never finish
try:
snmpEngine.transportDispatcher.runDispatcher()
except:
snmpEngine.transportDispatcher.closeDispatcher()
raise
Since the job never finishes, child process keeps running. I have to terminate it from the parent process since child process never terminates on its own. However, I want to send itemToSend to parent process before child process terminates. Can I return it to parent process somehow?
UPDATE: Let me explain how runDispatcher() of pysnmp module works
def runDispatcher():
while jobsArePending(): # jobs are always pending because of jobStarted() function
loop()
def jobStarted(jobId):
if jobId in jobs: #This way there's always 1 job remaining
jobs[jobId] = jobs[jobId] + 1
This is very frustrating. Instead of doing all this, is it possible to write an snmp trap listener on our own? Can you point me to the right resources?
The .runDispatcher() method actually invokes a mainloop of an asynchronous I/O engine (asyncore/twisted) which terminates as soon as no active pysnmp 'jobs' are pending.
You can make pysnmp dispatcher to cooperate with the rest of your app by registering your own callback timer function which will be invoked periodically from mainloop. In your callback function you could check if a termination event arrived and reset pysnmp 'job' what would make pysnmp mainloop to complete.
def timerCb(timeNow):
if terminationRequestedFlag: # this flag is raised by an event from parent process
# use the same jobId as in jobStarted()
snmpEngine.transportDispatcher.jobFinished(1)
snmpEngine.transportDispatcher.registerTimerCbFun(timerCb)
Those pysnmp jobs are just flags (like '1' in your code) that mean to tell I/O core that asynchronous applications still need this I/O core to run and serve them. Once the last of potentially many apps is no more interested in I/O core operation, the mainloop terminates.
If the child process may cooperate then you could use multiprocessing.Event to inform the child that it should exit and multiprocessing.Pipe could be used to send itemToSend to the parent:
#!/usr/bin/env python
import logging
import multiprocessing as mp
from threading import Timer
def child(stopped_event, conn):
while not stopped_event.wait(1):
pass
mp.get_logger().info("sending")
conn.send({'tosend': 'from child'})
conn.close()
def terminate(process, stopped_event, conn):
stopped_event.set() # nudge child process
Timer(5, do_terminate, [process]).start()
try:
print(conn.recv()) # get value from the child
mp.get_logger().info("received")
except EOFError:
mp.get_logger().info("eof")
def do_terminate(process):
if process.is_alive():
mp.get_logger().info("terminating")
process.terminate()
if __name__ == "__main__":
mp.log_to_stderr().setLevel(logging.DEBUG)
parent_conn, child_conn = mp.Pipe(duplex=False)
event = mp.Event()
p = mp.Process(target=child, args=[event, child_conn])
p.start()
child_conn.close() # child must be the only one with it opened
Timer(3, terminate, [p, event, parent_conn]).start()
Output
[DEBUG/MainProcess] created semlock with handle 139845842845696
[DEBUG/MainProcess] created semlock with handle 139845842841600
[DEBUG/MainProcess] created semlock with handle 139845842837504
[DEBUG/MainProcess] created semlock with handle 139845842833408
[DEBUG/MainProcess] created semlock with handle 139845842829312
[INFO/Process-1] child process calling self.run()
[INFO/Process-1] sending
{'tosend': 'from child'}
[INFO/Process-1] process shutting down
[DEBUG/Process-1] running all "atexit" finalizers with priority >= 0
[DEBUG/Process-1] running the remaining "atexit" finalizers
[INFO/MainProcess] received
[INFO/Process-1] process exiting with exitcode 0
[INFO/MainProcess] process shutting down
[DEBUG/MainProcess] running all "atexit" finalizers with priority >= 0
[DEBUG/MainProcess] running the remaining "atexit" finalizers
I just downloaded the mysql-proxy and created this script lua (found in Mysql docs):
function read_query(packet)
if string.byte(packet) == proxy.COM_QUERY then
print("QUERY: " .. string.sub(packet, 2))
end
end
This is the command-line I'm using:
mysql-proxy -P localhost:1234 -b localhost:3306 --proxy-lua-script=profile.lua --plugins=proxy
When I run a simple query (like "select * from table1"), this error is reported: "failed: .\lua-scope.c:241: stat(C:...\profile.lua) failed: No error (0)"
Note: If I run mysql-proxy without lua script, no error occurs.
I need to install something to get mysql-proxy and query tracing working?
My environment is Windows 7 Professional x64.
Sorry the bad english.
The error you're getting is caused by --proxy-lua-script pointing to a file that mysql-proxy can't find. Either you've typed the name in wrong, you've typed the path in wrong, or you are expecting it in your CWD and it's not there. Or actually, looking at the entire error a little more closely, it seems possible that mysql-proxy itself sees the file in CWD itself OK, but one of the underlying modules doesn't like it (possibly because mysql-proxy changes the CWD somehow?)
Try saving profile.lua to the root of your C: drive and trying different versions of the option like so:
--proxy-lua-script=c:\profile.lua
--proxy-lua-script=\profile.lua
--proxy-lua-script=/profile.lua
One of those would probably work
simple query log lua script:
require("mysql.tokenizer")
local fh = io.open("/var/log/mysql/proxy.query.log", "a+")
fh:setvbuf('line',4096)
local the_query = "";
local seqno = 0;
function read_query( packet )
if string.byte(packet) == proxy.COM_QUERY then
seqno = seqno + 1
the_query = (string.gsub(string.gsub(string.sub(packet, 2), "%s%s*", ' '), "^%s*(.-)%s*$", "%1"))
fh:write(string.format("%s %09d %09d : %s (%s) -- %s\n",
os.date('%Y-%m-%d %H:%M:%S'),
proxy.connection.server.thread_id,
seqno,
proxy.connection.client.username,
proxy.connection.client.default_db,
the_query))
fh:flush()
return proxy.PROXY_SEND_QUERY
else
query = ""
end
end