Hi i have little long time query which takes almost 400ms .
this is my query
SELECT id, `from`
FROM messages
WHERE `to` = ?
AND `to_viewed` = '0'
AND `to_deleted` = '0'
AND TIMESTAMPDIFF(SECOND,created,?)< 20 AND TIMESTAMPDIFF(SECOND,created,?)>= 0
How can i optimize this ?
obs i have no idex .
EDIT to show my EXPLAIN
id select_type table type possible_keys key key_len ref rows Extra
1 SIMPLE messages ALL created NULL NULL NULL 10 Using where
Make sure you have an index on the created column. Then use
AND created BETWEEN ? AND DATE_ADD(?, INTERVAL 20 SECOND);
An index can't be used when you perform a calculation on the column. So you should calculate the ends of the time range and then compare the column to that.
Related
I have a problem with the count() of paginate still in cakephp version 3.3:
My table has 6,000,000 records.
The fields involved here are name and cityf. Both have index in MySQL
I'm showing 10 and 10 and despite the query is very fast, the count() of paginate is taking more than 50 seconds.
How to solve this in version 3.3 of cakephp. Follows the two SQL statements and times below:
Select query main:
SELECT
Rr.id AS `Rr__id`,
Rr.idn AS `Rr__idn`,
Rr.aniver AS `Rr__aniver`,
Rr.pessoa AS `Rr__pessoa`,
Rr.name AS `Rr__name`,
Rr.phoner AS `Rr__phoner`,
Rr.tipolf AS `Rr__tipolf`,
Rr.addressf AS `Rr__addressf`,
Rr.num_endf AS `Rr__num_endf`,
Rr.complem AS `Rr__complem`,
Rr.bairrof AS `Rr__bairrof`,
Rr.cityf AS `Rr__cityf`,
Rr.statef AS `Rr__statef`,
Rr.cepf AS `Rr__cepf`,
Rr.n1 AS `Rr__n1`,
Rr.n2 AS `Rr__n2`,
Rr.smerc AS `Rr__smerc`,
Rr.n3 AS `Rr__n3`,
Rr.n4 AS `Rr__n4`,
Rr.fone AS `Rr__fone`,
Rr.numero AS `Rr__numero`
FROM
`MG` Rr
WHERE
(
Rr.name like 'MARCOS%'
AND Rr.cityf like 'BELO HORIZONTE%'
)
ORDER BY
name asc
LIMIT
10 OFFSET 0
= 10 ms
Explain:
id select_type table type possible_keys key key_len ref rows Extra
1 SIMPLE Rr range NAME,CITYF,cityfbairrof,cityfaddressf,cityfbairrofaddressf,namen1n2n3n4 NAME 63 NULL 21345 Using index condition; Using where
-
Select query count:
SELECT
(
COUNT(*)
) AS `count`
FROM
`MG` Rr
WHERE
(
Rr.name like 'MARCOS%'
AND Rr.cityf like 'BELO HORIZONTE%'
)
= 51.247 ms
Explain:
id select_type table type possible_keys key key_len ref rows Extra
1 SIMPLE Rr range NAME,CITYF,cityfbairrof,cityfaddressf,cityfbairrofaddressf,namen1n2n3n4 NAME 63 NULL 21345 Using index condition; Using where
It's happening in several other cases: Always count of query is very slow.
I appreciate any help.
Marcos
I have a question about this query and indexing
my query is :
EXPLAIN SELECT * FROM s1_training where amt > 0 LIMIT 500
and i indexed ( amt ) But its not effected if i have this wheres :
WHERE amt != 0
WHERE amt > 0
WHERE amt < 0
the result is :
id select_type table type possible_keys key key_len ref rows Extra
1 SIMPLE s1_training ALL amt NULL NULL NULL 64 Using where
The only way index will be used is to set
WHERE amt = number ( like amt = 2 )
and the result will be
id select_type table type possible_keys key key_len ref rows Extra
1 SIMPLE s1_training ref amt amt 4 const 1
thanks in advance for any help
best regards.
It depends on the number of rows and the cardinality of values.
For example if most values are >0 then mysql optimizer will decide to to a table scan since is faster.
For the second condition WHERE amt=2 this will probably return far fewer values so the index is used because is faster.
I have a query running on MySQL DB and is very slow.
Is there anyway I can optimize the following
SELECT mcm.merchant_name,
( ( Sum(ot.price) + Sum(ot.handling_charges)
+ Sum(ot.sales_tax_recd)
+ Sum(ot.shipping_cost) - Sum(ot.sales_tax_payable) ) -
Sum(im.break_even_cost) ) AS PL,
ot.merchant_id
FROM order_table ot,
item_master im,
merchant_master mcm
WHERE ot.item_id = im.item_id
AND ot.merchant_id = mcm.merchant_id
GROUP BY mcm.merchant_name
ORDER BY pl DESC
LIMIT 0, 10;
The Above Query is taking more than 200 seconds to execute.
Explain Result:
id select_type table type possible_keys key key_len ref rows Extra
1 SIMPLE ot ALL "merchant_id,item_id" NULL NULL NULL 507910 "Using temporary; Using filesort"
1 SIMPLE mcm eq_ref "PRIMARY,merchant_id" PRIMARY 4 stores.ot.merchant_id 1
1 SIMPLE im eq_ref "PRIMARY,item_id" PRIMARY 4 stores.ot.item_id 1
Also, I got Error-1003 when I run EXPLAIN EXTENDED
use mysql explain plan to find out why it is taking so long and then maybe create some indexes or change your code.
Update
Based upon this make sure you have an composite index on the order_table on merchant_id,item_id
I'm using MySQL with PDO in PHP and I have a SQL query, which works as expected. However, I care about performance and would like to know if I could improve my query. I'm also asking, because I want to gain some more background knowledge of SQL.
Let's say I have two tables that have a few equal fields (and some additional information, which are different in each table):
table `blog_comments`: id, userid (int) | timestamp (int) | content (varchar) | other
table `projects_comments`: id, userid (int) | timestamp (int) | content (varchar) | other
The field id is the primary key, userid + timestamp have an index in both tables, and timestamp is simply the unixtime with the length of 10 (integer).
As a simple spam protection, I block a user from submitting a new comment (no matter if blog, project or anything else) until 60 seconds have passed since his last comment. To achieve this, I get the latest timestamp of that user from all the comments tables.
This is my working query:
SELECT MAX(`last_timestamp`) AS `last_timestamp`
FROM
(
SELECT `userid`, max(`timestamp`) AS `last_timestamp`
FROM `blog_comments`
GROUP BY `userid`
UNION ALL
SELECT `userid`, max(`timestamp`) as `last_timestamp`
FROM `projects_comments`
GROUP BY `userid`
) AS `subquery`
WHERE `userid` = 1
LIMIT 0, 1;
As you can notice, I use GROUP BY inside the subqueries, and in the main query I simply filter the userid (in this case: 1). The advantage: I just need to pass the userid once as a parameter.
Now, I am interested into how SQL exactly works. I think it will be like this: SQL first performs the subqueries, groups all the existing rows by userid and returns the whole set to the main query, which then applies the where clause to find the required userid. This seems like a big leak of performance to me.
So I thought on slightly changing the query:
SELECT max(`last_timestamp`) AS `last_timestamp`
FROM
(
SELECT max(`timestamp`) AS `last_timestamp`
FROM `blog_comments`
WHERE `userid` = 1
UNION ALL
SELECT max(`timestamp`) as `last_timestamp`
FROM `projects_comments`
WHERE `userid` = 1
) AS `subquery`
LIMIT 0, 1
Now I have to pass the userid twice, and still the whole set of rows will be looked up for the given userid. I am not sure if this really improves the performance.
I don't have any large data amount yet to really test it, maybe I will do some test scenarios later. I would be really interested into knowing if there would be a difference, when there would be many data sets in those tables?
Would appreciate any ideas, information and tips, thanks in advance.
Edit:
MySQL explain of the first query:
id select_type table type possible_keys key key_len ref rows Extra
1 PRIMARY <derived2> ALL NULL NULL NULL NULL 4 Using where
2 DERIVED blog_comments range NULL userid 8 NULL 10 Using index for group-by
3 UNION projects_comments index NULL userid 12 NULL 6 Using index
NULL UNION RESULT <union2,3> ALL NULL NULL NULL NULL NULL
MySQL explain of the second query:
id select_type table type possible_keys key key_len ref rows Extra
1 PRIMARY <derived2> ALL NULL NULL NULL NULL 2
2 DERIVED NULL NULL NULL NULL NULL NULL NULL Select tables optimized away
3 UNION NULL NULL NULL NULL NULL NULL NULL Select tables optimized away
NULL UNION RESULT <union2,3> ALL NULL NULL NULL NULL NULL
As an alternative approach...
SELECT 'It''s been more than 1 minute since your last post' As result
WHERE NOT EXISTS (
SELECT *
FROM blog_comments
WHERE userid = 1
AND timestamp > Date_Sub(Current_Timestamp, INTERVAL 1 MINUTE)
)
AND NOT EXISTS (
SELECT *
FROM projects_comments
WHERE userid = 1
AND timestamp > Date_Sub(Current_Timestamp, INTERVAL 1 MINUTE)
)
There will be a result if userid = 1 hasn't got a timestamped record within the last minute in either table.
You can also swap the logic around...
SELECT 'You''re not allowed to post just yet...' As result
WHERE EXISTS (
SELECT *
FROM blog_comments
WHERE userid = 1
AND timestamp > Date_Sub(Current_Timestamp, INTERVAL 1 MINUTE)
)
OR EXISTS (
SELECT *
FROM projects_comments
WHERE userid = 1
AND timestamp > Date_Sub(Current_Timestamp, INTERVAL 1 MINUTE)
)
This second option will probably be more efficient (EXISTS vs NOT EXISTS) but that's for you to test and prove ;)
The answer to your question is that the second should perform better in MySQL than the first, for exactly the reason you gave. MySQL will run the full group by on all the data and then select the one group.
You can see the different in execution paths by putting an explain in front of the query. That will give ou some idea of what the query is really doing.
If you have an index on user_id, timestamp, then the second query will run quite fast, only using the index. Even without an index, the second query would do a full table scan of the two tables -- and that is it. The first will do a full table scan and a file sort for the aggregation. The second takes longer.
If you wanted to pass in the userid only once, you could do something like:
select coalesce(greatest(bc_last_timestamp, pc_last_timestamp),
bc_last_timestamp, pc_last_timestamp
)
from (select (SELECT max(`timestamp`) FROM `blog_comments` bc where bc.userid = const.userid
) bc_last_timestamp,
(SELECT max(`timestamp`) FROM `projects_comments` pc where pc.userid = const.userid
) pc_last_timestamp
from (select 1 as userid) const
) t;
The query looks arcane but it should optimize similarly to your second one.
SELECT count, item, itemid
FROM items
ORDER BY count DESC
LIMIT 20
Takes .0011
Explain:
id select_type table type possible_keys key key_len ref rows Extra
1 SIMPLE items index NULL count 4 NULL 20
I have indexes on itemid(primary key) and count (INDEX)
Does anyone have suggestions for how this could be better accomplished?
It seems like your long_query_time variable/setting is extremely short. The default is 10 seconds, but if your query is taking 0.0011 seconds, it obviously shouldn't be logged with the default setting. Try increasing it to something reasonable for your setup (1 second+ probably) and see if this still happens.