mysql_query on utf8_general_ci collation - mysql

I'm having some trouble with running a query on a database using utf8 encoding.
Everything works fine until I try a string with special characters.
I can't really make a lot of changes to the database since I'm adding some separate pages to an existing prestashop template.
This is my code (sorry, but some of it is in Romanian)
$acces_server=mysql_connect($db_host, $db_user, $db_pass);
if(!$acces_server) die ("Server-ul nu este disponibil momentan !");
mysql_set_charset('utf8');
echo mysql_client_encoding();
$acces_bd = mysql_select_db($db_name);
if (!$acces_bd) die ("Ne pare rau, dar baza de date nu poate fi accesata momentan !");
mysql_query('SET NAMES utf8') or die(mysql_error());
mysql_query('SET CHARACTER SET utf8') or die(mysql_error());
mysql_query('SET COLLATION_CONNECTION="utf8_general_ci" ') or die(mysql_error());
If I try to make a query like this I get 0 rows:
$experienta= $_GET['name'];
$rez=mysql_query("SELECT nume_cutie FROM rez_experiente WHERE nume = '".$experienta."'");
I've used echo to display the query and then typed it in phpMyAdmin and it worked .
Also the head section has this line :
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
I really don't know what else to do , all the solutions I've found so far don't seem to work. I have even tried using mysqli instead and the result was the same.
Please help me , and thank you for your time.
*Edit
I tried using stmt->prepare with mysqli and that didn't work.
I've also tried doing this :
$experienta2=str_replace("?","%",utf8_decode($experienta));
$rez=mysql_query("SELECT nume_cutie FROM rez_experiente WHERE nume LIKE '".$experienta2."'");
Wich turns this :
SELECT nume_cutie FROM rez_experiente WHERE nume LIKE '5th Avenue Beauty Center - Răsfăţ'
Into this:
SELECT nume_cutie FROM rez_experiente WHERE nume LIKE '5th Avenue Beauty Center - R%sf%%'
And again the query works in phpMyAdmin but mysql_query() returns nothing
I switched to utf8_unicode_ci and nothing changed.

After the database connection, run these queries:
SET character_set_client=utf8;
SET character_set_connection=utf8;
SET character_set_database=utf8;
SET character_set_results=utf8;
SET character_set_server=utf8;

Related

Perl issue when encoding mysql data from UTF-8 to UCS-2 for SMPP

I am trying to fetch UTF-8 accentuated characters "é" "ê" from mysql and convert them to UCS-2 when sending over SMPP. The data is stored as utf8_general_ci and I perform the following when opening the DB connection:
$dbh->{'mysql_enable_utf8'}=1;
$dbh->do("set NAMES 'utf8'");
If I test the sending part by hard coding the string value with "é" "ê" using data_encoding=8, it goes through perfectly. However if I comment out the first line and just use what comes from the DB, it fails. Also, if I try to send the characters using the DB and setting data_encoding=3, it also works fine, but then the "ê" would not appear, which is also expected. Here is what I use:
$fred = 'éêcole'; <-- If I comment out this line, the SMPP call fails
$fred = decode('utf-8', $fred);
$fred = encode('UCS-2', $fred);
$resp_pdu = $short_smpp->submit_sm(
source_addr_ton => 0x00,
source_addr_npi => 0x01,
source_addr => $didnb,
dest_addr_ton => 0x01,
dest_addr_npi => 0x01,
destination_addr => $number,
data_coding => 0x08,
short_message => $fred
) or do {
Log("ERROR: submit_sm indicated error: " . $resp_pdu->explain_status());
$success = 0;
};
The different values for the data_coding fields are the following:
Meaning of "data_coding" field in SMPP
00000000 (0) - usually GSM7
00000011 (3) for standard ISO-8859-1
00001000 (8) for the universal character set -- de facto UTF-16
The SMPP provider's documentation also mentions that special characters should be handled via UCS-2:
https://community.sinch.com/t5/SMS-365-enterprise-service/Handling-Special-Characters/ta-p/1137
How should I prepare the data that is coming out of the DB to make this SMPP call work?
I am using Perl v5.10.1
Thanks !
$dbh->{'mysql_enable_utf8'} = 1; is used to decode the values returned from the database, causing queries to return decoded text (strings of Unicode Code Points). It makes no sense to decode such a string. Go straight to the encode.
my $s_ucp = "\xE9\xEA\x63\x6F\x6C\x65"; # éêcole
# -or-
use utf8; # Script is encoded using UTF-8.
my $s_ucp = "éêcole";
printf "%vX\n", $s_ucp; # E9.EA.63.6F.6C.65
my $s_ucs2be = encode('UCS-2', $s_ucp);
printf "%vX\n", $s_ucs2be; # 0.E9.0.EA.0.63.0.6F.0.6C.0.65
SET NAMES says the encoding you have/want in the client. That is, regardless of the encoding in the table, MySQL will convert it to whatever SET NAMES says during a SELECT.
So, feed what comes from the SELECT directly to SMPP. (It won't be readable by most other clients.)
SET NAMES ucs2
(The collation is irrelevant to the encoding.)
You could ask the SELECT to convert with something like
CONVERT(col_name, CHAR UNICODE)
https://dev.mysql.com/doc/refman/8.0/en/cast-functions.html

Powershell array value in MySQL insert is outputting entire array

First off I am new to Powershell, as in this is my first ever attempt. I'm doing the I can program in other languages so I can hack through this project. I have spent a few days now trying to solve this one, and I know is something stupid. I'm just stuck.
# Insert into master table
$query = "INSERT INTO ``master``(``diff``, ``key``) VALUES ('$diff', '$key')"
Invoke-MySqlQuery -Query $query
$query
This works fine the test output displays:
INSERT INTO `master`(`diff`, `key`) VALUES ('248', 'k000002143200000000000000680006080005500900030082670009461000500000091000000000000')
It also inputs into the MySQL DB just fine.
This following the exact same format is not working for me. Like I said at the top, I know it is some stupid formatting thing I'm missing.
# Insert into answers table
$query = "INSERT INTO ``puzzles``(``id``, ``p1``, ``p2``, ``p3``, ``p4``, ``p5``, ``p6``, ``p7``, ``p8``, ``p9``, ``p10``, ``p11``, ``p12``, ``p13``, ``p14``, ``p15``, ``p16``, ``p17``, ``p18``, ``p19``, ``p20``, ``p21``, ``p22``, ``p23``, ``p24``, ``p25``, ``p26``, ``p27``, ``p28``, ``p29``, ``p30``, ``p31``, ``p32``, ``p33``, ``p34``, ``p35``, ``p36``, ``p37``, ``p38``, ``p39``, ``p40``, ``p41``, ``p42``, ``p43``, ``p44``, ``p45``, ``p46``, ``p47``, ``p48``, ``p49``, ``p50``, ``p51``, ``p52``, ``p53``, ``p54``, ``p55``, ``p56``, ``p57``, ``p58``, ``p59``, ``p60``, ``p61``, ``p62``, ``p63``, ``p64``, ``p65``, ``p66``, ``p67``, ``p68``, ``p69``, ``p70``, ``p71``, ``p72``, ``p73``, ``p74``, ``p75``, ``p76``, ``p77``, ``p78``, ``p79``, ``p80``, ``p81``) VALUES ('$id', '$p[0]', '$p[1]', '$p[2]', '$p[3]', '$p[4]', '$p[5]', '$p[6]', '$p[7]', '$p[8]', '$p[9]', '$p[10]', '$p[11]', '$p[12]', '$p[13]', '$p[14]', '$p[15]', '$p[16]', '$p[17]', '$p[18]', '$p[19]', '$p[20]', '$p[21]', '$p[22]', '$p[23]', '$p[24]', '$p[25]', '$p[26]', '$p[27]', '$p[28]', '$p[29]', '$p[30]', '$p[31]', '$p[32]', '$p[33]', '$p[34]', '$p[35]', '$p[36]', '$p[37]', '$p[38]', '$p[39]', '$p[40]', '$p[41]', '$p[42]', '$p[43]', '$p[44]', '$p[45]', '$p[46]', '$p[47]', '$p[48]', '$p[49]', '$p[50]', '$p[51]', '$p[52]', '$p[53]', '$p[54]', '$p[55]', '$p[56]', '$p[57]', '$p[58]', '$p[59]', '$p[60]', '$p[61]', '$p[62]', '$p[63]', '$p[64]', '$p[65]', '$p[66]', '$p[67]', '$p[68]', '$p[69]', '$p[70]', '$p[71]', '$p[72]', '$p[73]', '$p[74]', '$p[75]', '$p[76]', '$p[77]', '$p[78]', '$p[79]', '$p[80]')"
$query
$p[0]
$p[1]
$p[2]
$p[3]
$p[4]
$p[5]
Invoke-MySqlQuery -Query $query
This is the output I get on one of the runs:
INSERT INTO `puzzles`(`id`, `p1`, `p2`, `p3`, `p4`, `p5`, `p6`, `p7`, `p8`, `p9`, `p10`, `p11`, `p12`, `p13`, `p14`, `p15`, `p16`, `p17`, `p
18`, `p19`, `p20`, `p21`, `p22`, `p23`, `p24`, `p25`, `p26`, `p27`, `p28`, `p29`, `p30`, `p31`, `p32`, `p33`, `p34`, `p35`, `p36`, `p37`, `p
38`, `p39`, `p40`, `p41`, `p42`, `p43`, `p44`, `p45`, `p46`, `p47`, `p48`, `p49`, `p50`, `p51`, `p52`, `p53`, `p54`, `p55`, `p56`, `p57`, `p
58`, `p59`, `p60`, `p61`, `p62`, `p63`, `p64`, `p65`, `p66`, `p67`, `p68`, `p69`, `p70`, `p71`, `p72`, `p73`, `p74`, `p75`, `p76`, `p77`, `p
78`, `p79`, `p80`, `p81`) VALUES ('2596', '000002143200000000000000680006080005500900030082670009461000500000091000000000000[0]', '000002143
200000000000000680006080005500900030082670009461000500000091000000000000[1]', '0000021432000000000000006800060800055009000300826700094610005
00000091000000000000[2]', '000002143200000000000000680006080005500900030082670009461000500000091000000000000[3]', '0000021432000000000000006
80006080005500900030082670009461000500000091000000000000[4]', '00000214320000000000000068000608000550090003008267000946100050000009100000000
0000[5]', '000002143200000000000000680006080005500900030082670009461000500000091000000000000[6]', '00000214320000000000000068000608000550090
0030082670009461000500000091000000000000[7]', '000002143200000000000000680006080005500900030082670009461000500000091000000000000[8]', '00000
2143200000000000000680006080005500900030082670009461000500000091000000000000[9]', '000002143200000000000000680006080005500900030082670009461
000500000091000000000000[10]', '000002143200000000000000680006080005500900030082670009461000500000091000000000000[11]', '0000021432000000000
00000680006080005500900030082670009461000500000091000000000000[12]', '0000021432000000000000006800060800055009000300826700094610005000000910
00000000000[13]', '000002143200000000000000680006080005500900030082670009461000500000091000000000000[14]', '00000214320000000000000068000608
0005500900030082670009461000500000091000000000000[15]', '000002143200000000000000680006080005500900030082670009461000500000091000000000000[1
6]', '000002143200000000000000680006080005500900030082670009461000500000091000000000000[17]', '000002143200000000000000680006080005500900030
082670009461000500000091000000000000[18]', '000002143200000000000000680006080005500900030082670009461000500000091000000000000[19]', '0000021
43200000000000000680006080005500900030082670009461000500000091000000000000[20]', '0000021432000000000000006800060800055009000300826700094610
00500000091000000000000[21]', '000002143200000000000000680006080005500900030082670009461000500000091000000000000[22]', '00000214320000000000
0000680006080005500900030082670009461000500000091000000000000[23]', '00000214320000000000000068000608000550090003008267000946100050000009100
0000000000[24]', '000002143200000000000000680006080005500900030082670009461000500000091000000000000[25]', '000002143200000000000000680006080
005500900030082670009461000500000091000000000000[26]', '000002143200000000000000680006080005500900030082670009461000500000091000000000000[27
]', '000002143200000000000000680006080005500900030082670009461000500000091000000000000[28]', '0000021432000000000000006800060800055009000300
82670009461000500000091000000000000[29]', '000002143200000000000000680006080005500900030082670009461000500000091000000000000[30]', '00000214
3200000000000000680006080005500900030082670009461000500000091000000000000[31]', '00000214320000000000000068000608000550090003008267000946100
0500000091000000000000[32]', '000002143200000000000000680006080005500900030082670009461000500000091000000000000[33]', '000002143200000000000
000680006080005500900030082670009461000500000091000000000000[34]', '000002143200000000000000680006080005500900030082670009461000500000091000
000000000[35]', '000002143200000000000000680006080005500900030082670009461000500000091000000000000[36]', '0000021432000000000000006800060800
05500900030082670009461000500000091000000000000[37]', '000002143200000000000000680006080005500900030082670009461000500000091000000000000[38]
', '000002143200000000000000680006080005500900030082670009461000500000091000000000000[39]', '00000214320000000000000068000608000550090003008
2670009461000500000091000000000000[40]', '000002143200000000000000680006080005500900030082670009461000500000091000000000000[41]', '000002143
200000000000000680006080005500900030082670009461000500000091000000000000[42]', '000002143200000000000000680006080005500900030082670009461000
500000091000000000000[43]', '000002143200000000000000680006080005500900030082670009461000500000091000000000000[44]', '0000021432000000000000
00680006080005500900030082670009461000500000091000000000000[45]', '0000021432000000000000006800060800055009000300826700094610005000000910000
00000000[46]', '000002143200000000000000680006080005500900030082670009461000500000091000000000000[47]', '00000214320000000000000068000608000
5500900030082670009461000500000091000000000000[48]', '000002143200000000000000680006080005500900030082670009461000500000091000000000000[49]'
, '000002143200000000000000680006080005500900030082670009461000500000091000000000000[50]', '000002143200000000000000680006080005500900030082
670009461000500000091000000000000[51]', '000002143200000000000000680006080005500900030082670009461000500000091000000000000[52]', '0000021432
00000000000000680006080005500900030082670009461000500000091000000000000[53]', '0000021432000000000000006800060800055009000300826700094610005
00000091000000000000[54]', '000002143200000000000000680006080005500900030082670009461000500000091000000000000[55]', '00000214320000000000000
0680006080005500900030082670009461000500000091000000000000[56]', '00000214320000000000000068000608000550090003008267000946100050000009100000
0000000[57]', '000002143200000000000000680006080005500900030082670009461000500000091000000000000[58]', '000002143200000000000000680006080005
500900030082670009461000500000091000000000000[59]', '000002143200000000000000680006080005500900030082670009461000500000091000000000000[60]',
'000002143200000000000000680006080005500900030082670009461000500000091000000000000[61]', '0000021432000000000000006800060800055009000300826
70009461000500000091000000000000[62]', '000002143200000000000000680006080005500900030082670009461000500000091000000000000[63]', '00000214320
0000000000000680006080005500900030082670009461000500000091000000000000[64]', '00000214320000000000000068000608000550090003008267000946100050
0000091000000000000[65]', '000002143200000000000000680006080005500900030082670009461000500000091000000000000[66]', '000002143200000000000000
680006080005500900030082670009461000500000091000000000000[67]', '000002143200000000000000680006080005500900030082670009461000500000091000000
000000[68]', '000002143200000000000000680006080005500900030082670009461000500000091000000000000[69]', '0000021432000000000000006800060800055
00900030082670009461000500000091000000000000[70]', '000002143200000000000000680006080005500900030082670009461000500000091000000000000[71]',
'000002143200000000000000680006080005500900030082670009461000500000091000000000000[72]', '00000214320000000000000068000608000550090003008267
0009461000500000091000000000000[73]', '000002143200000000000000680006080005500900030082670009461000500000091000000000000[74]', '000002143200
000000000000680006080005500900030082670009461000500000091000000000000[75]', '000002143200000000000000680006080005500900030082670009461000500
000091000000000000[76]', '000002143200000000000000680006080005500900030082670009461000500000091000000000000[77]', '0000021432000000000000006
80006080005500900030082670009461000500000091000000000000[78]', '0000021432000000000000006800060800055009000300826700094610005000000910000000
00000[79]', '000002143200000000000000680006080005500900030082670009461000500000091000000000000[80]')
0
0
0
0
0
2
In the $query variable the $p[X] does not work like the $p[X] below. It's spitting out the entire array. Obviously it's not working when it sends it to MySQL, I get the ID correct and that's it.
Thank you all for the help in advance, thumping my head on the keyboard hurts after awhile!
Ian
Bonus question! Any tips on this would be great as well. Any of the 0s don't need to be there, I actually want them as NULL. Any tips on a clean way to do that would be wonderful. Full disclosure I have not even looked into this part yet!
EDIT - In response to #Mathias R. Jessen
I removed all the other code not having to do with $p, to make it easy on the eyes.
# Set the key variable
$key = (Get-Content Puzzles.txt)[$loop] # ....73.146...8.......6......5..2..8.8..5.9..1.923..........273.........8..3..1.46 #Extreme
$key = $key.Replace(".", "0") # 000073014600080000000600000050020080800509001092300000000002730000000008003001046 #Extreme
$key = -join('k', $key.Substring(0,81)) # k000073014600080000000600000050020080800509001092300000000002730000000008003001046
$check = (Invoke-MySqlQuery -Query "SELECT COUNT(*) FROM ``master`` WHERE ``key`` = '$key'")
$check = $check.'COUNT(*)'
# Check in master table if key exists, i.e. not a new puzzle
IF ($check -eq 0) {
$p = $key.Substring(1,81) # 000073014600080000000600000050020080800509001092300000000002730000000008003001046
# Insert into answers table
$query = "INSERT INTO ``puzzles``(``id``, ``p1``, ``p2``, ``p3``, ``p4``, ``p5``, ``p6``, ``p7``, ``p8``, ``p9``, ``p10``, ``p11``, ``p12``, ``p13``, ``p14``, ``p15``, ``p16``, ``p17``, ``p18``, ``p19``, ``p20``, ``p21``, ``p22``, ``p23``, ``p24``, ``p25``, ``p26``, ``p27``, ``p28``, ``p29``, ``p30``, ``p31``, ``p32``, ``p33``, ``p34``, ``p35``, ``p36``, ``p37``, ``p38``, ``p39``, ``p40``, ``p41``, ``p42``, ``p43``, ``p44``, ``p45``, ``p46``, ``p47``, ``p48``, ``p49``, ``p50``, ``p51``, ``p52``, ``p53``, ``p54``, ``p55``, ``p56``, ``p57``, ``p58``, ``p59``, ``p60``, ``p61``, ``p62``, ``p63``, ``p64``, ``p65``, ``p66``, ``p67``, ``p68``, ``p69``, ``p70``, ``p71``, ``p72``, ``p73``, ``p74``, ``p75``, ``p76``, ``p77``, ``p78``, ``p79``, ``p80``, ``p81``) VALUES ('$id', '$p[0]', '$p[1]', '$p[2]', '$p[3]', '$p[4]', '$p[5]', '$p[6]', '$p[7]', '$p[8]', '$p[9]', '$p[10]', '$p[11]', '$p[12]', '$p[13]', '$p[14]', '$p[15]', '$p[16]', '$p[17]', '$p[18]', '$p[19]', '$p[20]', '$p[21]', '$p[22]', '$p[23]', '$p[24]', '$p[25]', '$p[26]', '$p[27]', '$p[28]', '$p[29]', '$p[30]', '$p[31]', '$p[32]', '$p[33]', '$p[34]', '$p[35]', '$p[36]', '$p[37]', '$p[38]', '$p[39]', '$p[40]', '$p[41]', '$p[42]', '$p[43]', '$p[44]', '$p[45]', '$p[46]', '$p[47]', '$p[48]', '$p[49]', '$p[50]', '$p[51]', '$p[52]', '$p[53]', '$p[54]', '$p[55]', '$p[56]', '$p[57]', '$p[58]', '$p[59]', '$p[60]', '$p[61]', '$p[62]', '$p[63]', '$p[64]', '$p[65]', '$p[66]', '$p[67]', '$p[68]', '$p[69]', '$p[70]', '$p[71]', '$p[72]', '$p[73]', '$p[74]', '$p[75]', '$p[76]', '$p[77]', '$p[78]', '$p[79]', '$p[80]')"
Invoke-MySqlQuery -Query $query
}
You need to use the subexpression operator $() to evaluate the collection at that index during string interpolation. Without this operator, the content of your entire collection is being printed, as well as your literal index syntax.
Your first example works as expected because you're only interpolating simple variables, without doing any additional work to them.
Here's a simple example from the command line:
C:\> $arr = 1,2,3,4
Outside of a string:
C:\> $arr[0]
1
During string interpolation, without the subexpression operator:
C:\> "$arr[0]"
1 2 3 4[0]
During string interpolation, with the subexpression operator:
C:\> "$($arr[0])"
1
This means that your example would become something like this:
...VALUES ('$id', '$($p[0])', '$($p[1])'...
Note that $id is working correctly because it is a simple variable. You only need to use the subexpression operator for additional work like evaluating indexes, properties, etc.
This concept is also sometimes called variable expansion, if you would like to research further.

PREG REPLACE ERROR ALPHA NUMERIC STRING - MYSQL

Hi I used the preg_replace for replace the non alpha numeric data with ''.
1)
SELECT
PREG_REPLACE('[^A-Za-z0-9]','',CATALOGUEREFERENCE) AS INSTRUMENT
FROM
FILES;
2)
select
preg_replace('[[:alnum:]]', '',CATALOGUEREFERENCE) as instrument
from
files;
First query displays the records as blob and the second query displays null records. Can anyone suggest the query modification for replacing the non alpha numeric data with ''.
Input file :
GETA2000003890
MAC00007000
NAS00006000
MAS000007000
MAS2SA200006000
Desired output:
GET
MAC
NAS
MAS
MAS2SA2
Thanks a lot
I think your desired output is: GET MAC NAS MAS MAS.
Code:
$input = 'GETA2000003890 MAC00007000 NAS00006000 MAS000007000 MAS2SA200006000';
$output = preg_replace('/([A-Z]{3})(\w*)/i', '$1', $input);
echo $output; // GET MAC NAS MAS MAS

Mysql-Html string problem

I have a table, in joomla I'm getting the following data:
1. fc k��ln - vfl wolfsburg
germany 1. bundesliga
so I created a test page, I set up the header as utf-8, and everything was fine, I received the wanted data:
Array
(
[0] => Array
(
[id] => 4e36e64eb34d2
[team1] => 1. FC Köln
[team2] => VFL Wolfsburg
[league] => Germany 1. Bundesliga
[sport] => Soccer
[time] => 2011-08-06 15:30:00
)
)
I also check the joomla header and it's contains the charset:
<meta http-equiv="content-type" content="text/html; charset=utf-8" />
What am I doing wrong?
your database connection has to be set UTF8
set the mysql connection charset in includes/database.php file at about line 102 (second line below)
$this->_table_prefix = $table_prefix;
//#mysql_query("SET NAMES 'utf8'", $this->_resource); // THIS IS THE LINE TO UNCOMMENT
$this->_ticker = 0;
$this->_og = array();
Ofcourse your data in table has to be utf-8, check this.
Other possibible thing to check is if your browser sees this page as utf8 (in firefox: View->character encoding)
Maybe your files are in wrong encoding? I was having similar problem whan my files were in CP-1251, all data were in UTF-8 and I needed to show danish symbols. Changing file-encoding has solved this problem. Try, maybe it will be helpfull for you too.

How to log error queries in mysql?

I know that there is general_log that logs all queries, but I want to find out which query has an error, and get the error message. I have tried running an error query on purpose, but it logs as a normal query and doesn't report it with error. Any ideas?
There is no functionality in MySQL to do this.
You will have to stick with application-layer logging.
I know this is ancient now, but for anyone having the same issue that lands here from Google, here's my two cents.
If you're using the cli client, you can simply redirect your sterr to a file, and then parse through that.
mysql -u user -p 2> errors.log
It is possible to log error queries with the MariaDB Audit Plugin.
The MariaDB Audit Plugin works for MariaDB, MySQL and Percona Server.
For example, for these queries
select now();
select now()+();
select 9+();
select 'hello';
log seems like this:
20150807 23:00:36,mnv-Satellite-L300D,root,localhost,82,377,QUERY,`test`,'select now()
LIMIT 0, 1000',0
20150807 23:00:37,mnv-Satellite-L300D,root,localhost,82,379,QUERY,`test`,'select now()+()',1064
20150807 23:00:37,mnv-Satellite-L300D,root,localhost,82,382,QUERY,`test`,'select 9+()',1064
20150807 23:00:38,mnv-Satellite-L300D,root,localhost,82,383,QUERY,`test`,'select \'hello\'
LIMIT 0, 1000',0
Last column is return code. 0 is Ok. Else - error.
Even though this question is quite old I hope it will be useful to someone who searched for mysql log error queries or similar terms.
Not too long ago I also required mysqld to log only erroneous queries. I found that mysql-proxy enables you to do that and wrote a small LUA script:
local err_flag = false
function read_query( packet )
if packet:byte() == proxy.COM_QUERY then
local user = proxy.connection.client.username
local host = proxy.connection.client.src.name
if user:lower() == 'someuser' then -- change this to any condition where errors should be logged
proxy.queries:append(1, packet, {resultset_is_needed = true})
proxy.queries:append(2, string.char(proxy.COM_QUERY) .. "SET #last_query = '" .. string.sub(packet, 2) .. "'", {resultset_is_needed = true} )
proxy.queries:append(3, string.char(proxy.COM_QUERY) .. "SHOW WARNINGS", {resultset_is_needed = true} )
end
return proxy.PROXY_SEND_QUERY
end
end
function insert_query(err_t, err_n, err_m)
local query = "INSERT INTO `somedb`.`mysql_error` " .. -- change log destination
"(`date`, `err_num`,`err_type`, `err_message`, `problem_query`, `conn_id`)" ..
" VALUES ( NOW(), " ..
err_n .. "," .. "\"" ..
err_t .."\"" .. "," .. "\"" ..
err_m .. "\"" .. "," ..
"#last_query" .. "," ..
proxy.connection.server.thread_id .. ")"
proxy.queries:append(4, string.char(proxy.COM_QUERY) .. query, {resultset_is_needed = true})
return proxy.PROXY_SEND_QUERY
end
function read_query_result(inj)
local res = assert(inj.resultset)
if inj.id == 1 then
err_flag = false
if res.query_status == proxy.MYSQLD_PACKET_ERR then
err_flag = true
return proxy.PROXY_IGNORE_RESULT
end
elseif inj.id == 2 then
return proxy.PROXY_IGNORE_RESULT
elseif inj.id == 3 then
if err_flag == true then
for row in res.rows do
proxy.response.type = proxy.MYSQLD_PACKET_ERR
proxy.response.errmsg = row[3]
insert_query(row[1], row[2], row[3])
end
return proxy.PROXY_SEND_RESULT
end
return proxy.PROXY_IGNORE_RESULT
elseif inj.id == 4 then
return proxy.PROXY_IGNORE_RESULT
end
end
DDL needed for logging table, adjust somedb.mysql_error to liking, but don't forget to do so in the above LUA script also.
CREATE TABLE `somedb`.`mysql_error` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`date` datetime NOT NULL,
`err_num` smallint(6) NOT NULL,
`err_type` varchar(10) COLLATE utf8_unicode_ci NOT NULL,
`err_message` varchar(255) COLLATE utf8_unicode_ci NOT NULL,
`problem_query` varchar(8000) COLLATE utf8_unicode_ci NOT NULL,
`conn_id` int(11) NOT NULL,
PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci
To use the script, run
/path/to/mysql-proxy --proxy-lua-script=/path/to/mysql-proxy-log-error-queries.lua
or if that fails (>=v0.9)
/path/to/mysql-proxy --proxy-lua-script=/path/to/mysql-proxy-log-error-queries.lua --plugins=proxy
The proxy runs on port 4040 by default, to test:
mysql -u username -p --host=127.0.0.1 --port=4040
and run some faulty sql.
When everything seems to be in order set the port in your application to 4040 instead of the actual mysqld port and you have mysql error logging on the database level.
Final note: mysql-proxy is beta. Use with caution I guess. Been running here for almost half a year now without problems, however YMMV.
I have tried running an error query on
purpose, but it logs as a normal query
and doesn't report it with error. Any
ideas?
so, you did it wrong. No other idea without the code.
in PHP I'm doing it this way (assume you're using mysql driver):
$res=mysql_query($sql) or trigger_error(mysql_error().$sql);
it will log all erroneous queries if you have log_errors setting on (and you have to)
EDIT:
I see now, you want global level logging,m not application level.
But may be application level will suit you as well?
MariaDB is capable to do this with a plugin https://mariadb.com/kb/en/sql-error-log-plugin/ which is distributed with mariadb.
I tried this yesterday and it works as advertised.
One need to just run:
install plugin SQL_ERROR_LOG soname 'sql_errlog';
and the queries with errors will go to $datadir/sql_errors.log that is on most linux install at /var/lib/mysql/sql_errors.log.