It is a purely MySQL tweaking question on logging the failed SQL attempts.
I need a solution to log the failed SQLs on the MySQL database.
The MySQL general query log does not work here, because it will log only the successful queries only. I need to know the failed ones in a different log file.
My scenario is: I need to cleanup a big messy SQLs and PHP Scripts (literally, around 7,000 SQLs being executed per page). Since the errors are suppressed on the server, even the SQLs failed queries will continue and show the good contents on the website. But I am sure, there are a lot of SQL errors. I found 2 links but cannot retry. This one and writing server side audit plugin in MySQL.
Is using Kontrolbase an answer as mentioned here.
Percona Server 5.5.37 and 5.6.17 include an implementation of an audit plugin which is free and open-source. Read about it here: http://www.percona.com/doc/percona-server/5.6/management/audit_log_plugin.html
The audit plugin logs both successful queries and erroneous queries. Here's an example where I queried my table foo and then a non-existent table foobar:
<AUDIT_RECORD
"NAME"="Query"
"RECORD"="489_2014-05-23T14:06:09"
"TIMESTAMP"="2014-05-23T14:08:13 UTC"
"COMMAND_CLASS"="select"
"CONNECTION_ID"="32709"
"STATUS"="0"
"SQLTEXT"="select * from foo"
"USER"="root[root] # localhost []"
"HOST"="localhost"
"OS_USER"=""
"IP"=""
/>
<AUDIT_RECORD
"NAME"="Query"
"RECORD"="490_2014-05-23T14:06:09"
"TIMESTAMP"="2014-05-23T14:08:19 UTC"
"COMMAND_CLASS"="select"
"CONNECTION_ID"="32709"
"STATUS"="1146"
"SQLTEXT"="select * from foobar"
"USER"="root[root] # localhost []"
"HOST"="localhost"
"OS_USER"=""
"IP"=""
/>
STATUS=1146 is indicated for the error query above. See http://dev.mysql.com/doc/refman/5.6/en/error-messages-server.html to reference server errors. For example, 1146 is ER_NO_SUCH_TABLE.
If you can't use this solution, then you'll have to change your PHP code to remove the error suppression, and log the errors.
Related
How do I view MySQL query errors and warnings in RDS? I am still using MySQL 5.6.
I have looked in the log files (under Log & Events), but they remain empty even when I do a query that generates an error in the console. For example, I ran a query that gave me error 1054 (unknown column), but nothing appeared in the error log. I've realized that this isn't the type of errors that get logged.
The log_warnings parameter is set to 1. I've tried setting general_log=1, but that generates way to much info, and still doesn't log the errors.
The reason that I need this is that I am upgrading from 5.6 to 5.7. I need like to see what warnings I'm currently getting since some warning will become errors in 5.7, and need to be fixed.
You can run SHOW WARNINGS immediately after the query, to get all conditions (errors, warnings, and notes). You have to do this in the client that ran the query. It's not a log.
There is no log for error, warnings, or notes for each query. The errors and warnings in the MySQL Server error log are for server errors, not for individual queries.
Queries that can't be parsed are not written to the general log or the slow query log by default. If you set the global option log_raw=1 then the general log will log erroneous queries, but still won't log the error itself. Even with that option, the slow query log won't log queries that can't be parsed.
A free tool from Percona may be helpful for your testing. pt-upgrade allows you to use a query log file as input, and it runs the query against two instances of MySQL, and reports any difference in errors or warnings, or results.
I thought of another partial solution: the PERFORMANCE_SCHEMA has tables for statement events. The statement event tables have columns for error number and message caused by each query, if any. But only a count of the warnings, not the warning message(s) themselves.
I just upgraded MySQL from 5.1 to 5.5.
I fixed few issues running mysql_upgrade, and changing some deprecated configurations...
I also updated PHP, from 5.3.3-7 to 5.3.29-1.
But, since that, I'm having a reccurent problem (always thrown in this order) :
1. Client* - PHP Warning
Warning: Packets out of order. Expected 1 received 0. Packet size=1 in
/home/www/www.mywebsite.com/shared/vendor/doctrine/dbal/lib/Doctrine/DBAL/Connection.php
line 694
2. Client* - PHP Warning
Warning: PDOStatement::execute() [pdostatement.execute]: Error reading
result set's header in
/home/www/www.mywebsite.com/shared/vendor/doctrine/dbal/lib/Doctrine/DBAL/Connection.php
line 694
3. Server* - MySQL Warning :
150127 17:25:15 [Warning] Aborted connection 309 to db:
'my_database' user: 'root' host: '127.0.0.1' (Got an error
reading communication packets)
4. Client* - PHP Error
PDOStatement::execute() [pdostatement.execute]: MySQL server
has gone away in
/home/www/www.mywebsite.com/shared/vendor/doctrine/dbal/lib/Doctrine/DBAL/Connection.php
line 694
*NB: What I call "Client" is the PHP Application, and "Server" is the MySQL Server, even if they're both on the same localhost Server.
So, apparently, the origin of all those problems is the first one : "Packets out of order".
But when I search for this error I can't find many answers, and they are most of the time not related to my problem : I use Doctrine as an abstraction, so I don't write any query or fetch any result myself. Plus, it's almost never the same values as me, but in my case I always get those values ("Expected 1 received 0. Packet size=1").
The closest result would be this MySQL bug report, but "No feedback was provided for this bug for over a month, so it is
being suspended automatically"...
Plus, some of the "2." errors aren't thrown by my PHP Doctrine code (they're not executed from my localhost, but from another known external service, probably using some old PHP Propel code).
So that might mean there is a problem with my MySQL configuration itself, but I tried changing some parameters without obtaining any obvious effect (sometimes it takes more time after restarting MySQL to get the first errors for example).
Any help would be very much appreciated !
And here is my current configuration (I've got 2 MySQL instances, the second one using replication is mostly for read only).
I also checked most of the system resources with Munin and didn't see anything abnormal (the RAM usage for example is pretty high, but as there is 50Go on the server it's not full at all).
UPDATE
I isolated an SQL query that was repeatedly failing from my PHP Client. When I executed from my local with MySQL Workbench, it did exactly the same (closed the connexion with a MySQL server has gone away message). When I did it from the sql command line it also did the same. Then I executed it from the sql command line on the server host, and it succeded. But some time after when I tried again from Workbench/whatever it worked... So it looks like those "corrupted packets" are cached and disapear after some time.
Thanks, I fixed this issue doing :
RESET QUERY CACHE;
FLUSH QUERY CACHE;
I need to capture all of the query errors that are happening on my MySQL server. I've looked into the general_log and that appears to just save the queries, but NOT the result (whether there was an error thrown or not). I've heard some people say the server can't do this, and it can only be done on the client. This seems like a common function, so I would be surprised if there was no option to log query errors on the server.
How can I log these query errors?
There is no error.log in MySQL like you desire, see here what you can log with MySQL.
The only way is to look in your own error.logs e.g. in php:
catch the mysql_error or mysqli_error in php and write your own log-file
or
look in your apache-error-log when you didn´t catch your errors
I got problem (#2006 Mysql server gone away) with mysql while connecting and performing some operations through web browser.
Operation Listed below:
When Executing big procedure
Importing database dump
When Access some particular tables It immediately throws "Server gone away".
Refer this question for Scenarios: Record Not Inserted - #2006 Mysql server gone away
Note : The above operations are works fine when I perform through terminal.
I tried some configuration as googing stated. That is set wait_timeout, max_allowed_packet. I checked for the bin_log but it is not available.
But the issues will not rectified.
What is the problem & How can I figure out & fix the issue?
what is the different between access phpmyadmin mysql server from web browser and terminal?
Where I can find the mysql server log file?
Note: If you know about any one of the above questions. Please post here. It would be helpful to trace.
Please help me to figure this out..
Thanks in advance...
Basically nothing except phpMyAdmin is limited by PHP's timeout and resource limits (limits to keep a runaway script from bogging down your entire machine for all eternity; see the docs for details of those values. In some cases, you might be authenticating through a different user account (for instance, root#localhost and root#127.0.0.1 aren't the same user), but as long as you're using a user with the same permissions the differences are minimal.
You can read more about logs in the MySQL manual, note that "By default, no logs are enabled (except the error log on Windows)".
Below are answer for question
From my research the problem is that browser have some limit to disconnect the connection i.e timeout connection. So that the above problem raised.
To resolve this problem
Go to /opt/lampp/phpmyadmin and open config.inc.php
add the command $cfg['ExecTimeLimit'] = 0;
Restart the xamp server. Now you can perform any operations.
`
2. Web client is differ from terminal because Terminal client will not getting timeout. Terminal client maintain the connection till the progress completed. I recommenced to use command prompt to import/export/run process by safe way.
Basically phpmyadmin will not have any log file. If you wanna see warnings and error you should configure the log file.
Configuration steps:
Go to /opt/lampp/etc/my.cnf
Add log_bin = /opt/lampp/var/mysql/filename.log
Restart the xamp server. You can get the log information.
Is it possible to log CREATE / ALTER statements issued on a MySQL server through phpMyAdmin? I heard that it could be done with a trigger, but I can't seem to find suitable code anywhere. I would like to log these statements to a table, preferably with the timestamp of when they were issued. Can someone provide me with a sample trigger that would enable me to accomplish this?
I would like to log these statements so I can easily synchronize the changes with another MySQL server.
There is a patch for phpMyAdmin which provides configurable logging with only some simple code modifications.
We did this at my work and then i tweaked it further to log into folders by day, log IP addresses and a couple other things and it works great.
Thanks #Unreason for the link, i couldn't recall where i found it.
Here is a script that would do what you want for mysql-proxy (check the link on official docs how to install the proxy).
To actually log the queries you can use something as simple as
function string.starts(String,Start)
return string.sub(String,1,string.len(Start))==Start
end
function read_query( packet )
if string.byte(packet) == proxy.COM_QUERY then
local query = string.lower(string.sub(packet, 2))
if string.starts(query, "alter") or string.starts(query, "create") then
-- give your logfile a name, absolute path worked for me
local log_file = '/var/log/mysql-proxy-ddl.log'
local fh = io.open(log_file, "a+")
fh:write( string.format("%s %6d -- %s \n",
os.date('%Y-%m-%d %H:%M:%S'),
proxy.connection.server["thread_id"],
query))
fh:flush()
end
end
end
The script was adopted from here, search for 'simple logging'.
This does not care about results - even if the query returned an error it would be logged (there is 'more customized logging' example, which is a better candidate for production logging).
Also, you might take another approach if it is applicable for you - define different users in your database and give DDL rights only to a certain user, then you could log everything for that user and you don't have to worry about details (for example - proxy recognizes the following server commands, out of which it inspects only Query)
Installing the proxy is straight forward, when you test it you can run it with
mysql-proxy --proxy-lua-script=/path/to/script.lua
It runs on port 4040 by default so test it with
mysql -u user -p -h 127.0.0.1 -P 4040
(make sure you don't bypass the proxy; for example on my distro mysql -u user -p -h localhost -P 4040 completely ignored the port and connected over socket, which left me puzzled for a few minutes)
The answer to your question will fall into one of the listed in MySQL Server logs
If you just want to get the CREATE/ALTER statements, I would go with the general query log. But you will have to parse the file manually. Be aware of the security issues this approach raises.
In your scenario, replication seems to be an overkill.
Triggers are not a valid option since they are only supported at SELECT, UPDATE and INSERT level and not ALTER/CREATE.
Edit 1:
The query log would be the best choice but as you mentioned on busy servers the logs would cause a considerable efficiency penalty. The only additional alternative I know of is MySQL Proxy.
I think that your best bet would be to look at the use of stored procedures and functions here to make changes to your DB. That way you could look at manually logging data.