I am running below query on MySQL 5.7.22 and I see that queries are truncated under query column of the table.
select * from sys.`statements_with_temp_tables`
where db = 'schema_name'
order by memory_tmp_tables desc;
Is there any way we can retrieve these queries in full?
I have been enoyed by this query truncation for a long time too. So I just finished a short research to finally get a solution...
So. This 'truncation' is controlled by #sys.statement_truncate_len option. You can set it to reasonably long value before you select from sys.statements_with_temp_tables (or any other table containing monitoring data):
SET #sys.statement_truncate_len = 1000;
select * from sys.`statements_with_temp_tables`;
You can read more on this in official doc:
https://dev.mysql.com/doc/refman/5.7/en/sys-sys-config.html
P.S. If you use MySql Workbench: do not forget to set 'Max.Field Value Length to display' limit in Edit - Preferences - SQL Execution (main menu)
Related
I am facing a performance issue that has manifested itself after a migration from mysql 5.5 to mariadb 10.2.13 . The sizing of both machines including configs is the same. We have noticed that a certain select statement suddenly takes ~20 secs to execute instead of 0.5 sec. The data is purely read from memory as there is zero read IO to be measured during execution. Changing the optimizer_switch on the mariadb to the settings of mysql5.5 did not change the execution plan.
The explain for the select on the mariadb looks as follows:
The explain for the select on the mysql5.5 :
Select-Statement:
SELECT acmh.OBJ_VERSION,acmh.peannr,acmh.pzsrnr,xuts.ptitle,
acmh.pcategory,acmh.psalutatio,acmh.plastname,acmh.pfirstname,
acmh.pcompanyna,syoq.paddress1,syoq.pzip,syoq.pcity,syoq.pphone1,
syoq.pphone2,syoq.pphone3,syoq.pphone4,syoq.pemail,acmh.piscompany,
xuts.id
FROM catalog1.tr_table_acmh acmh,catalog1.tr_table_syoq syoq,
catalog1.tr_table_xuts xuts,catalog1.tr_table_wdvi link0,
catalog1.tr_table_wdvi acl0
WHERE acmh.id=syoq.id
AND acmh.id=xuts.id
AND syoq.id=xuts.id
AND xuts.OBJ_TYPE IN (1557)
AND ( (xuts.id=link0.pchild
AND link0.pparent='xkgrrslqkeaaaaendrxa'
AND link0.pname='folder')
and 1 = 1 )
AND xuts.pvcurrent=1
AND (acl0.pchild=xuts.pacl
AND acl0.pparent='xkgrrswxbjaaaaaaabip'
AND acl0.pvalue>=20
)
order by acmh.plastname asc,
acmh.pfirstname asc,
acmh.pcompanyna asc
LIMIT 10 OFFSET 0;
Is there any switch that im missing that could enforce a different behaviour on the mariadb?
Updating the statistics for the query optimizer (ANALYZE TABLE on the 3 tables involved) fixed the issue in this case.
The new execution plan for the SELECT now looks like this:
These may help performance:
link0: INDEX(pchild, pparent, pname)
link0: INDEX(pparent, pname, pchild)
xuts: INDEX(pvcurrent, OBJ_TYPE, id)
acl0: INDEX(pchild, pparent, pvalue)
You seem to have several tables that are 1:1; is this true? If so, why?
What is the datatype of id? It seems to be a strange length: 20.
I have the following query
SELECT t.res, IF(t.res=0, "zero", "more than zero")
FROM (
SELECT table.*, IF (RAND()<=0.2,1, IF (RAND()<=0.4,2, IF (RAND()<=0.6,3,0))) AS res
FROM table LIMIT 20) t
which returns something like this:
That's exactly what you would expect. However, as soon as I remove the LIMIT 20 I receive highly unexpected results (there are more rows returned than 20, I cut it off to make it easier to read):
SELECT t.res, IF(t.res=0, "zero", "more than zero")
FROM (
SELECT table.*, IF (RAND()<=0.2,1, IF (RAND()<=0.4,2, IF (RAND()<=0.6,3,0))) AS res
FROM table) t
Side notes:
I'm using MySQL 5.7.18-15-log and this is a highly abstracted example (real query is much more difficult).
I'm trying to understand what is happening. I do not need answers that offer work arounds without any explanations why the original version is not working. Thank you.
Update:
Instead of using LIMIT, GROUP BY id also works in the first case.
Update 2:
As requested by zerkms, I added t.res = 0 and t.res + 1 to the second example
The problem is caused by a change introduced in MySQL 5.7 on how derived tables in (sub)queries are treated.
Basically, in order to optimize performance, some subqueries are executed at different times and / or multiple times leading to unexpected results when your subquery returns non-deterministic results (like in my case with RAND()).
There are two easy (and likewise ugly) workarounds to get MySQL to "materialize" (aka return deterministic results) these subqueries: Use LIMIT <high number> or GROUP BY id both of which force MySQL to materialize the subquery and return the expected results.
The last option is turn off derived_merge in the optimizer_switch variable: derived_merge=off (make sure to leave all the other parameters as they are).
Further readings:
https://mysqlserverteam.com/derived-tables-in-mysql-5-7/
Subquery's rand() column re-evaluated for every repeated selection in MySQL 5.7/8.0 vs MySQL 5.6
I'm using the Wordnet SQL database from here: http://wnsqlbuilder.sourceforge.net
It's all built fine and users with appropriate privileges have been set.
I'm trying to find synonyms of words and have tried to use the two example statements at the bottom of this page: http://wnsqlbuilder.sourceforge.net/sql-links.html
SELECT synsetid,dest.lemma,SUBSTRING(src.definition FROM 1 FOR 60) FROM wordsXsensesXsynsets AS src INNER JOIN wordsXsensesXsynsets AS dest USING(synsetid) WHERE src.lemma = 'option' AND dest.lemma <> 'option'
SELECT synsetid,lemma,SUBSTRING(definition FROM 1 FOR 60) FROM wordsXsensesXsynsets WHERE synsetid IN ( SELECT synsetid FROM wordsXsensesXsynsets WHERE lemma = 'option') AND lemma <> 'option' ORDER BY synsetid
However, they never complete. At least not in any reasonable amount of time and I have had to cancel all of the queries. All other queries seem to work find and when I break up the second SQL example, I can get the individual parts to work and complete in reasonable times (about 0.40 seconds)
When I try and run the full statement however, the MySQL command line client just hangs.
Is there a problem with this syntax? What is causing it to take so long?
EDIT:
Output of "EXPLAIN SELECT ..."
Output of "EXPLAIN EXTENDED ...; SHOW WARNINGS;"
I did more digging and looking into the various statements used and found the problem was in the IN command.
MySQL repeats the statement for every single row in the database. This is the cause of the hang, as it had to run through hundreds of thousands of records.
My remedy to this was to split the command into two separate database calls first getting the synsets, and then dynamically creating a bound SQL string to look for the words in the synsets.
I've had a look and reviewed the MySQL syntax, but can't figure out the right query modifier.
When you perform a normal select query, if there are a lot of columns (or one or two wide columns) the output wraps around the screen making it hard to read.
+----------------------------------------+-----------+--------+-----------------------
-----------------+----------------------------------------+------+-------------------+
--------------+---------------------------+-----------------+-------------------+-----
-------------------+-----------------+-------------------+-----------+
I recall someone showing me a long time there was a way of coercing MySQL to output each column on its own line like so:
id: 123456
Short_Field: Boy it's short
A_Long_Field: This is quite long, wow, very lorum, much ipsum...
Can anyone tell me what I'd need to put in the follow query to replicate this behaviour:
SELECT * FROM TABLE WHERE COLUMN="value" LIMIT 1;
The mode you are looking for is called "vertical output" and it is available when using \G as the delimiter in the MySQL command line client.
SELECT * FROM `table` WHERE `column` = 'value' LIMIT 1\G
Documentation on MySQL command line options
I have a MySQL server as a linked server in Microsoft SQL Server 2008. For the link I use MySQL ODBC Connector version 5.1.8. When invoking queries using OPENQUERY (the only way I found of performing queries), problems occur. Simple queries, such as
SELECT * FROM OPENQUERY(MYSQL, 'SHOW TABLES')
work fine. Selection of individual columns, e.g.,
SELECT * FROM OPENQUERY(MYSQL, 'SELECT nr FROM letter')
works fine as well, but SELECT * syntax does not work. The query:
SELECT * FROM OPENQUERY(MYSQL, 'SELECT * FROM mytable')
raises an error:
Msg 7347, Level 16, State 1, Line 6
OLE DB provider 'MSDASQL' for linked
server 'MYSQL' returned data that does
not match expected data length for
column '[MSDASQL].let_nr'. The
(maximum) expected data length is 40,
while the returned data length is 0.
How can I make the SELECT * syntax work?
This problem happens if you are querying a MySQL linked server and the table you query has a datatype char(). This means fixed length and NOT varchar(). This happens when your fixed length field has a shorter string than the maximum length that SQL Server expected to get from the ODBC.
To fix this go to your MySQL server and change the datatype to varchar() leaving the length as it is. Example change char(10) to varchar(10).
Executing the following command before queries seems to help:
DBCC TRACEON(8765)
The error messages go away and queries seem to be working fine.
I'm not sure what it does though; I found it here: http://bugs.mysql.com/bug.php?id=46857
Strangely, SQL Server becomes unstable, stops responding to queries and finally crashes with scary-looking dumps in the logs a few minutes after several queries to the MySQL server. I am not sure if this has to do anything with the DBCC command, so I'm still interested in other possible solutions to this problem.
What I did to fix this since I can't modify the MySQL database structure is just create a view with a cast ex: CAST(call_history.calltype AS CHAR(8)) AS Calltype,
and select my view from MSSQL in my linked server.
The reason behind is that some strange types don't work well with the linked server (in my case the MySQL enum)
I found this
"The problem is that one of the fields
being returned is a blank or NULL CHAR
field. To resolve this in the Mysql
ODBC settings select the option "Pad
CHAR to Full Length"
Look at the last post here
An alternative would be to use the trim() function in your SELECT statement within OPENQUERY. The downside is you have to list each field individually, but what I did was create a view that calls OPENQUERY and then perfrom select * on the view.
Not ideal, but better than changing data types on tables!
Here is a crappy solution I came up with because I am unable to change the datatype to varchar as the db admin for the MySQL server is afraid it will cause issues with his scripts.
in my MySQL select query I run a case statement checking the character length of the string and add a filler character in front of the string "filling it up" to the max (in my case its a char(6)). then in the select statement of the openquery I strip the character back off.
Select replace(gradeid,'0','') as gradeid from openquery(LINKEDTOMYSQL, '
SELECT case when char_length(gradeid) = 0 then concat("000000", gradeID)
when char_length(gradeID) = 1 then concat("00000", gradeID)
when char_length(gradeID) = 2 then concat("0000", gradeID)
when char_length(gradeID) = 3 then concat("000", gradeID)
when char_length(gradeID) = 4 then concat("00", gradeID)
when char_length(gradeID) = 5 then concat("0", gradeID)
else gradeid end as gradeid
FROM sometableofmine')
it works but it probably is slower...
maybe you can make a MySQL function that will do the same logic, or come up with a more elegant solution.
I had the similar problem to this myself, which I resolved by wrapping the column-names in single ` style quotes.
Instead of...
column_name
...use...
`column_name`
Doing this helps the MySql query-engine should the column-name clash with a key or reserved-word.*
Instead of using SELECT * FROM TABLE_NAME, try to use all column names with quotes:
SELECT `column1`, `column2`, ... FROM TABLE_NAME
Example for normal datatype columns
SELECT * FROM OPENQUERY(MYSQL, 'SELECT `column1`, `column2`,...,`columnN` FROM mytable')
Example for ENUM datatype columns
SELECT * FROM OPENQUERY(MYSQL, 'SELECT `column1`, trim(`column2`) `column2`, `column3`,...,`columnN` FROM mytable')
*For those used to Sql Server, it is the MySql equivalent of wrapping a value in square-brackets, [ and ].