We are using MySQL InnoDB.
We have a query looks like this.
In our live environment, this query took more than 30 seconds to complete.
select count(*) as aggregate
from `parents`
where exists (
SELECT *
from `childs`
where `parents`.`id` = `childs`.`parent_id`
and exists (
SELECT *
from `users`
where `childs`.`user_id` = `users`.`id`
and `id` = '123456' )
and `status` = 'OK' )
And so we have exported the whole database and import into to our local mysql database.
And surprisingly, it took almost instant to get the same query results.
As so we suspect the table was not optimized and we have done the following.
optimize table users;
optimize table parents;
optimize table childs;
Unfortunately the query speed didn't improve.
Can anyone see what could goes wrong?
And why does export/import in local (with exactly same structure data) have almost instant query and the live took almost 30-60 seconds to complete?
EXPLAIN on both local and live shows a difference,
one of the DEPENDENT SUBQUERY for possible keys relating the parents and child table shows
Using where; FirstMatch(closing_batches)
but the live shows only Using where without the FirstMatch.
You can actually probably get all the data from a single query without even using the parents or user table -- IF the "Status" field is in the childs table.
From basic Transitive association,
if A = B and B = C, then A = C.
You are joining from Child to User by ID, then looking at the User ID = "123456".
This is the same as just asking for Childs.User_ID = "123456".
Likewise, from the Child joined to the parent by the Child.Parent_ID, it looks like your query is trying to get a count of distinct parent IDs that are associated with given childs.
So, the following SHOULD be able to get what you need.
select
count( distinct c.Parent_id ) Aggregate
from
childs c
where
c.user_id = '123456'
AND c.status = 'OK'
if the status field is on the PARENT table, you will need to join to that
select
count( distinct c.Parent_id ) Aggregate
from
childs c
join parents p
on c.parent_id = p.id
AND p.status = 'OK'
where
c.user_id = '123456'
For performance, I would ALSO have an index on the childs table on ( user_id, parent_id ). This can significantly optimize the query too.
This is probably equivalent:
select count(*) as aggregate
from `parents` AS p
where exists (
SELECT *
from `childs` AS c
JOIN users AS u ON c.user_id = u.id
WHERE c.user_id = 123456
AND p.`id` = c.`parent_id`
and `status` = 'OK'
)
OPTIMIZE TABLE is rarely useful.
Which table is status in?
Related
I have 3 tables - one for users, one for their incoming payments, and one for their outgoing payments. I want to display all incoming and outgoing payments in a single result set. I can do this with multiple selects and a union but it seems cumbersome, and I suspect its slow due to the subqueries - and the tables are extremely large (though I am using indexes). Is there a faster way to achieve this? Maybe using a full outer join?
Here is a simplified version of the schema with some example data:
create table users (
id int auto_increment,
name varchar(20),
primary key (id)
) engine=InnoDB;
insert into users (name) values ('bob'),('fred');
create table user_incoming_payments (
user_id int,
funds_in int
) engine=InnoDB;
insert into user_incoming_payments
values (1,100),(1,101),(1,102),(1,103),
(2,200),(2,201),(2,202),(2,203);
create table user_outgoing_payments (
user_id int,
funds_out int
) engine=InnoDB;
insert into user_outgoing_payments
values (1,100),(1,101),(2,200),(2,201);
And here is the ugly looking query which generates the result I want for user bob:
select * from (
(select u.name, i.funds_in, 0 as 'funds_out' from users u
inner join user_incoming_payments i on u.id = i.user_id)
union
(select u.name, 0 as 'funds_in', o.funds_out from users u
inner join user_outgoing_payments o on u.id = o.user_id)
) a where a.name = 'bob'
order by a.funds_in asc, a.funds_out asc;
And here is as close as I can get to doing the same thing with joins - its not correct though because I want this result set to look the same as the previous and I wasn't sure how to use full outer join:
select *
from users u
right join user_incoming_payments i on u.id = i.user_id
right join user_outgoing_payments o on u.id = o.user_id
where u.name = 'bob';
SQL Fiddle here
MySQL doesn't support FULL OUTER JOIN. Even if it did support it, I don't think you would want that, as it would introduce a semi-cartesian product... with each row from incoming_ matching every row in outgoing_, creating extra rows.
If there were four rows from incoming_ and six rows from outgoing_, the set produced by a join operation would contain 24 rows.
This really looks more like you want a set concatenation operation. That is, you have two separate sets that you want to concatenate together. That's not a JOIN operation. That's a UNION ALL set operation.
SELECT ... FROM ...
UNION ALL
SELECT ... FROM ...
If you don't need to remove duplicates (and it looks like you wouldn't want to in this scenario, if there are multiple rows in incoming_ with the same value of funds_in, I don't think you want to remove any of the rows.)...
Then use the UNION ALL set operator which does not perform the check for and removal of duplicate rows.
The UNION operator removes duplicate rows. Which (again) I don't think you want.
The derived table isn't necessary.
And MySQL doesn't "push" the predicate from the outer table into the inline view. Which means that MySQL is going to materialized a derived table with all incoming and outgoing for all users. And the the outer query is going to look through that to find the rows. And until the most recent versions of MySQL, there were no indexes created on derived tables.
See the answer from Strawberry for an example of a more efficient query.
With the small example set, indexes aren't going to make any difference. With a large set, however, you are going to want to add appropriate covering indexes.
Also, with queries like this, I tend to include a discriminator column that tells me which query returned a row.
(
SELECT 'i' AS src
, ...
FROM ...
)
UNION ALL
(
SELECT 'o' AS src
, ...
FROM ...
)
ORDER BY ...
With this model, I'd probably write that query as follows, but I doubt it makes much difference...
select u.name
, i.funds_in
, 0 funds_out
from users u
join user_incoming_payments i
on u.id = i.user_id
where u.name = 'bob'
union all
select u.name
, 0 funds_in
, o.funds_out
from users u
join user_outgoing_payments o
on u.id = o.user_id
where u.name = 'bob'
order
by funds_in asc
, funds_out asc;
However, note that there's no PK here, which may prove problematic.
If it was me, I'd have one table for transactions, which would include a transaction_id PK, a timestamp for each each transaction, and a column to record whether a value was a credit or a debit.
Hello everyone I am working on phpmyadmin database. Whenever I try to execute query it takes too much time more than 10 mins to show results. Is there any way to speed it up. please response.
The query is
SELECT ib.*, b.brand_name, m.model_name,
s.id as sale_id, br.branch_code,br.branch_name,r.rentry_date,r.id as rid
from in_book ib
left join brand b on ib.brand_id=b.id
left join model m on ib.vehicle_id=m.id
left join re_entry r on r.in_book_id=ib.id
left join sale s on ib.id=s.in_book_id
left join branch br on ib.branch_id=br.id
where ib.id !=''
and ib.branch_id='65'
group by ib.id
order by r.id ASC,
count(r.in_book_id) DESC ,
ib.purchaes_date ASC,
ib.id ASC
there are almost 7 tables
make sure you got an index on every key you use to join the tables.
from http://dev.mysql.com/doc/refman/5.5/en/optimization-indexes.html:
The best way to improve the performance of SELECT operations is to create indexes on one or more of the columns that are tested in the query. The index entries act like pointers to the table rows, allowing the query to quickly determine which rows match a condition in the WHERE clause, and retrieve the other column values for those rows. All MySQL data types can be indexed.
.. this of course also applies to the JOIN conditions.
You don't list any such indexes, however, I would start with the following suggested indexes
table index
in_book ( branch_id, id, brand_id, vehicle_id )
brand ( id, brand_name )
model ( id, model_name )
re_entry ( in_book_id, id, reentry_date )
sale ( in_book_id, id )
branch ( id )
Also, with MySQL, you can use a special keyword "STRAIGHT_JOIN" which tells the engine to query in the order you have selected the tables... Although you are doing LEFT JOINs, I don't think it will matter as it appears the secondary tables are all lookup type of tables and in_book is your primary. But as just a try it would be..
SELECT STRAIGHT_JOIN (...rest of query...)
I was trying to cross reference 2 tables using the following query from myPhpAdmin app:
select A.*
from purchases A
where A.user in (
select B.user
from users B
where B.ppi = 'Facebook Ads'
)
It accepted the syntax but the DB never returned. The users table is not small, 200k rows, but i run querys on it all the time so it shouldn't take that long.. Any ideas as to why this might not work? The query was stuck in the state:Sending data. I had to kill it because my database was broken at this point so I cannot run any other checks on this now and Im scared to try again :)
Running on mysql FYI.
What I really wanted was just to be able to operate on values in table purchases only when the same user id is present in the other table with the given ppi value.
Use a join and make sure, you have indices on B.user and B.ppi.
SELECT A.*
FROM purchases A
INNER JOIN users B ON A.user=B.user
WHERE B.ppi = 'Facebook Ads'
A join should be faster than a subquery. Try this:
Select A.* from purchases A
INNER JOIN users B on A.user = B.user
WHERE B.ppi='Facebook Ads'
EDITED:
So this is the query I use:
SELECT *
FROM contacts
WHERE id in
(
SELECT DISTINCT contacts.id
from contacts INNER
JOIN contacts2tags ON contacts.id = contacts2tags.contactid
WHERE ( tagid in ( 178 ) ) )
It runs very slow. Suggestions to optimise it?
I have added indexes but it still needs improvement!
contacts table contains id, first_name, last_name, ..and tags table contains id, name. contacts2tags table contains contactid and tagid which are same as contacts.id and tags.id respectively
EXPLAIN:
Please have a look at: optimise mysql query with LIKE operator for 10k records
It was stupid of me to post a portion of the query here. Sorry about that :P
I just compared queries
select benchmark(1000000, 1 in(1,2,3,4,5)); took 0.122 sec
select benchmark(1000000, (1=1 or 1=2 or 1=3 or 1=4 or 1=5)); took 0.088 sec
So can you try using or instead of in i am not sure but for in index is also not used so just give a try.
Use or instead of IN.
Your query should be:
SELECT c.id FROM contacts c inner join contacts2tags t on c.id = t.contactid WHERE ( t.tagid =7 or t.tagid =4) GROUP BY c.id HAVING count(distinct t.tagid) = 2
The plan you've shown us says that the optimizer only expects to have to fetch 3 rows of data. If its "slow" (you didn't say how slow) then presumably the plan is not optimal or your data is very skewed.
The first thing I'd check is whether the index stats are up to date. Then I'd try...
SELECT count(*)
FROM contacts c
inner join contacts2tags t
on c.id = t.contactid
WHERE t.tagid in (7,4)
To find out how many rows the DBMS is actually processing to resolve the query. If this number is very high then you're unlikely to get much of a performance improvement from tuning the query.
....but one potential solution would be....
SELECT c.id
FROM contacts c,
contacts2tags t4,
contacts2tags t7,
WHERE c.id = t4.contactid
AND c.id = t7.contactid
AND t4.contactid=t7.contactid
AND t4.tagid=4
AND t7.tagid=7;
Can't see the need for using a sub query in your updated query. Try the following (although preferably use the names of the required columns from contacts rather than contacts.*).
SELECT DISTINCT contacts.*
FROM contacts
INNER JOIN contacts2tags ON contacts.id = contacts2tags.contactid
WHERE tagid IN ( 178 )
Make sure there is an index on tagid on the contacts2tags table.
If contactid / tagid on contacts2tags is unique (which I would hope) then no need for the DISTINCT. This can be enforced by adding a unique covering index on tagid / contactid (in that order to allow this key to be used in the WHERE clause).
I was wondering why you would first JOIN contacts and contacts2tags in a subquery and then use that to filter contacts again !?!
I presume the reason is that you already optimised the DISTINCT onto the id (lean) id field instead of doing it afterwards on the entire contacts.* record.
Anyway, what you actually should do IMHO is the following:
SELECT *
FROM contacts
WHERE EXISTS ( SELECT *
FROM contacts2tags
WHERE contacts2tags.contactid = contacts.id
AND tagid in ( 178 ) )
This will NOT double contacts records as the query processor knows it can stop searching once it finds a first hit in the contacts2tags table.
As for using IN() vs using OR... I doubt it will make a lot of difference as internally it probably gets translated to the same thing anyway (?!)
I'm stuck on a MySQL query. I have a temporary orders table with the following structure:
session_id
product_id
product_type -- 'institute','state','region','country'
For all institutes, states, regions and countries I have individual tables.
I want to create a MySQL query which fetches the data from my temp table and makes the join with the corresponding table depending upon product_type field.
If I use left join with 5 tables or use union it could be a really time consuming task; so I was looking for something different.
I would advise checking the answers in this question as they seem to match your specific problem https://stackoverflow.com/a/9327678/1213554
The short version though is that in order to be able to efficiently perform this request a database restructuring may well be required I'm afraid.
What you are looking for specifically is not possible, you'll have to use a UNION to do something along the lines of the following. As you say it will be a time consuming task though.
(
SELECT tempData.*
FROM tempData
INNER JOIN institutes
ON institutes.id = tempData.product_id
WHERE tempData.product_type = 'institute'
) UNION (
SELECT tempData.*
FROM tempData
INNER JOIN states
ON states.id = tempData.product_id
WHERE tempData.product_type = 'state'
) UNION (
SELECT tempData.*
FROM tempData
INNER JOIN regions
ON regions.id = tempData.product_id
WHERE tempData.product_type = 'region'
) UNION (
SELECT tempData.*
FROM tempData
INNER JOIN countries
ON countries.id = tempData.product_id
WHERE tempData.product_type = 'country'
)