I have this query:
SELECT
a.id,
a.name,
count(b.id),
count(c.id),
count(e.id),
count(f.id)
FROM
organizations a
LEFT JOIN vessels b ON a.id = b.organization_id
LEFT JOIN licences c ON a.id = c.organization_id
LEFT JOIN fleets e ON a.id = e.organization_id
LEFT JOIN users f ON a.id = f.organization_id
GROUP BY a.id;
In all tables there's a proper index (on the primary index, and organization_id), there's about 80 rows in organizations, 400 in fleets, 2900 in vessels, 3000 in licences and 10 in users
This query doesn't even succeed, it's stuck on copying to temp table
How should I re-work this query to make it work (fast) ?
id select_type table type possible_keys key key_len ref rows Extra
1 SIMPLE a index PRIMARY 4 1
1 SIMPLE b ref organisation_id organisation_id 4 fuel.a.id 70 Using index
1 SIMPLE c ref organisation_id organisation_id 4 fuel.a.id 15 Using index
1 SIMPLE e ref organisation_id organisation_id 4 fuel.a.id 5
1 SIMPLE f ref organization_id organization_id 5 fuel.a.id 1 Using index
Your joins do not depend on each other, that's why the temp tables are exploding.
A simple fix is to make:
SELECT
a.id,
a.name,
(select count(*) from vessels b where a.id = b.organization_id group by b.organization_id),
(select count(*) from licenses b where a.id = b.organization_id group by b.organization_id),
(select count(*) from fleets b where a.id = b.organization_id group by b.organization_id),
(select count(*) from users b where a.id = b.organization_id group by b.organization_id),
FROM
organizations a
It will be far more faster, if you do it like this:
SELECT
a.id,
a.name,
v.total,
w.total,
x.total,
y.total
FROM
organizations a
LEFT JOIN (select b.organizantion_id, count(*) total from vessels b group by b.organization_id) v on v.organization_id=a.id
LEFT JOIN (select b.organizantion_id, count(*) total from licenses b group by b.organization_id) w on w.organization_id=a.id
LEFT JOIN (select b.organizantion_id, count(*) total from fleets b group by b.organization_id) x on x.organization_id=a.id
LEFT JOIN (select b.organizantion_id, count(*) total from users b group by b.organization_id) y on y.organization_id=a.id
Related
My question is about how to optimize and reduce size of a sql query. I want to join more than 20 multiple queries using UNION, it is giving me the correct result as per the below logic, but I am looking for two things here
something more efficient
I already have 20 UNIONS in my query, and every month I have to add 2-4 UNIONS more which make this query very long so is there any way this query can be rephrased with less code
Select
'343' As 'Manual ID',
'24/07/2020' As 'Date',
A.ID,
O.Order_Name,
C.Customer_Name,
Q.Quantity
From Shipper A
Left Join Order O A.ID = O.ID
Left Join Customer C A.ID = C.ID
Left Join Quantity Q Q.ID = C.ID
where A.ID IN (1)
UNION
Select
'323' As 'Manual ID',
'24/08/2020' As 'Date',
A.ID,
O.Order_Name,
C.Customer_Name,
Q.Quantity
From Shipper A
Left Join Order O A.ID = O.ID
Left Join Customer C A.ID = C.ID
Left Join Quantity Q Q.ID = C.ID
where A.ID IN(2,3,4)
and so on ...
Result
Manual ID | Date | Shipper | Order Name | Customer Name | Qty
343 | 24/07/2020 | 1 | order1 | A | 5
323 | 24/08/2020 | 2 | order2 | B | 2
323 | 24/08/2020 | 3 | order3 | C | 1
323 | 24/08/2020 | 4 | order4 | D | 12
You can try this:
Select
CASE
WHEN A.ID IN(1) THEN '343'
WHEN A.ID IN(2,3,4) THEN '323'
END As 'Manual ID',
CASE
WHEN A.ID IN(1) THEN '24/07/2020'
WHEN A.ID IN(2,3,4) THEN '24/08/2020'
END As 'Date',
A.ID,
O.Order_Name,
C.Customer_Name,
Q.Quantity
From Shipper A
Left Join Order O A.ID = O.ID
Left Join Customer C A.ID = C.ID
Left Join Quantity Q Q.ID = C.ID
Where A.ID IN(1,2,3,4)
First suggestion is to move the parameters in to another table, then join on it. You can even make that an inline view if you don't want to use a real table...
Second suggestion is to use UNION ALL to avoid the costs of deduplication incurred by UNION.
SELECT
params.*,
O.Order_Name,
C.Customer_Name,
Q.Quantity
FROM
(
SELECT '343' As 'Manual ID', '24/07/2020' As 'Date', 1 AS A_ID
UNION ALL SELECT '323' As 'Manual ID', '24/08/2020' As 'Date', 2 AS A_ID
UNION ALL SELECT '323' As 'Manual ID', '24/08/2020' As 'Date', 3 AS A_ID
UNION ALL SELECT '323' As 'Manual ID', '24/08/2020' As 'Date', 4 AS A_ID
)
AS params
INNER JOIN Shipper A ON A.ID = params.A_ID
Left Join Order O ON A.ID = O.ID
Left Join Customer C ON C A.ID = C.ID
Left Join Quantity Q ON Q.ID = C.ID
Alternatively, don't recompute this every month. Write a new query each month, and insert the results into another table?
If you just want to go for query the better way would be to use the case when statement but every now and then you need to keep updating the query adding new cases.
Another, optimized solution will be to create a new table to store
Manual ID, Date, (Common) ID present in Shipper (Table). Then create a view to join all above tables with new Table.
New Table
Manual ID | Date | ID |
343 | 24/07/2020 | 1 |
323 | 24/08/2020 | 2 |
323 | 24/08/2020 | 3 |
323 | 24/08/2020 | 4 |
Then Create a View Joining all Tables including new new table with ID.
In this you just need add new value to new table and you will complete result in view it self.
CREATE VIEW MY_VIEW
AS
SELECT * FROM
(
Select
T.[Manual ID],
T.[Date],
A.ID,
O.Order_Name,
C.Customer_Name,
Q.Quantity
From Shipper A
Left Join Order O A.ID = O.ID
Left Join Customer C A.ID = C.ID
Left Join Quantity Q Q.ID = C.ID
Left Join NewTable T T.ID = A.ID
)
Now just insert value in new table and fetch complete data from MY_VIEW. It will give the same result as you are excepting.
I have a table addresses and many tables that has a address_id column that references an address (all have foreign key contraints). I want to find the number of addresses that are not referenced from any of the other tables.
I've tried the following query, but it is VERY slow:
SELECT COUNT(*)
FROM addresses A
LEFT JOIN customers C ON C.address_id = A.id
LEFT JOIN agreements AG ON AG.address_id = A.id
LEFT JOIN products P ON P.address_id = A.id
LEFT JOIN letters L ON L.address_id = A.id
WHERE C.id IS NULL
AND AG.id IS NULL
AND P.id IS NULL
AND L.id IS NULL
Is there any way I can do this, without the query taking forever?
I would start by rewriting this with not exists:
select count(*)
from addresses a
where
not exists (select 1 from customers c where c.address_id = a.id)
and not exists (select 1 from agreements g where g.address_id = a.id)
and not exists (select 1 from products p where p.address_id = a.id)
and not exists (select 1 from letters l where l.address_id = a.id)
Then, make sure to have the following indexes in place to speed up the query:
customers(address_id)
agreements(address_id)
products(address_id)
letters(address_id)
If you have properly defined the address_id columns as foreign keys to address(id), then these indexes are already there.
I have two table 1 is account and the other one is account_spend_history AS ash account is the parent/master table and the shin a sub table. and has a relationship OneToMany with account(account table id is a foreign key in ash as account_id). please see the image
Now I need to get the account.id, total spend (which is the sum of the amount_spend with same account_id) and last spend (is the last record inserted in ash table against and account_id i.e. amount_spend value corresponding with MAX(ash.id)), that is
id | spend_total | last_spend
---------------------------------
1 | 30 | 18
2 | 280 | 120
3 | 20 | 20
SELECT a.id, SUM(ash.amount_spend) AS spend_total
FROM accounts as a
INNER JOIN account_spend_history AS ash ON ash.account_id = a.id
GROUP BY a.id
Im getting the account.id and sum of ash.amount spend, but I also need last spend. How to get that?
Thanks.
Here's one option using a correlated subquery:
SELECT a.id, SUM(ash.amount_spend) AS spend_total,
(SELECT amount_spend
FROM account_spend_history as ash2
WHERE ash2.account_id = a.id
ORDER BY ash2.id DESC LIMIT 1) as last_spend
FROM accounts as a
INNER JOIN account_spend_history AS ash ON ash.account_id = a.id
GROUP BY a.id
You can get the MAX(ash.id) as well in your query and then use it to join back to account_spend_history table:
SELECT id, spend_total, amount_send as last_spend
FROM (
SELECT a.id, SUM(ash.amount_spend) AS spend_total, MAX(ash.id) AS ash_id
FROM accounts as a
INNER JOIN account_spend_history AS ash ON ash.account_id = a.id
GROUP BY a.id) AS t1
INNER JOIN account_spend_history AS ash ON ash.id = t1.ash_id
You can use a in clause and subquery
SELECT a.id, SUM(ash.amount_spend) AS spend_total, x.last_spend
FROM accounts as a
INNER JOIN account_spend_history AS ash ON ash.account_id = a.id
INNER JOIN (
select account_id, last_spend, start_date
from account_spend_history
where (start_date, acoount_id) in (
select max(start_date), account_id
from account_spend_history
group by account_id
)
) x on x.id = a.id
GROUP BY a.id
I am trying to figure out how to find duplicats based on several different columns and tables.
I've got these tables:
products
tags (lists tagid and productid - one row per tag)
groups (lists groupid and productid - one row per groupid)
I want to find exact matches in my table products on columns productName, brandid, origin. But to cast the row as a duplicate I also need to compare so that they have the exact same tags (column: tagid) and groups (column: groupid) assigned.
Every product may have multiple tags and multiple groups.
This is what I've come up with... but it's not quite doing what I need it to.
SQLFiddle
http://sqlfiddle.com/#!9/43f19/1
In my SQL fiddle example I have listed 10 different products.
For example, products 1,2 are exact matches and thus should be listed as a duplicate.
Product number 3 only has one group assigned and thus differ from product 1 and 2 even if any other parameter fits (it should not be listed). My intention with the dupid column would be to list the first entry of a set of duplicates.
id | name | brandid | origin | tags | groups | dupid
1 | prod | 1 | England | 1,2 | 1,2 | 1
2 | prod | 1 | England | 1,2 | 1,2 | 1
3 | prod | 1 | England | 1,2 | 1 | 3
Complete set of items that should be listed as exact matches in my SQL fiddle are:
id 1
id 2
id 4
id 5
My guess why this fails is that I have not succeeded to involve the tags and the groups correctly into my comparison.
SELECT m.*,dup.id AS dupid,GROUP_CONCAT(DISTINCT t.tagid ORDER BY t.tagid ASC) AS alltags,GROUP_CONCAT(DISTINCT g.groupid ORDER BY g.groupid ASC) AS groups
FROM `products` m
JOIN (SELECT id,`productName`, brandid, origin, COUNT(*) AS c FROM products
GROUP BY `productName`, brandid, origin HAVING c > 1) dup ON m.`productName` = dup.`productName` AND m.brandid = dup.brandid AND m.origin = dup.origin
LEFT JOIN tags AS t ON t.productid = m.id
LEFT JOIN groups AS g ON g.productid = m.id
GROUP BY m.id
ORDER BY `productName`,brandid,origin
Any help and/or advice on how to achieve this is highly appricated.
My guess is that you are missing an aggregation function on the subquery on ID field, also - you need to group by productname,origin and brand and not id so try this:
SELECT m.*,dup.id AS dupid,GROUP_CONCAT(DISTINCT t.tagid ORDER BY t.tagid ASC) AS alltags,GROUP_CONCAT(DISTINCT g.groupid ORDER BY g.groupid ASC) AS groups
FROM `products` m
JOIN (SELECT min(id) as id,`productName`, brandid, origin, COUNT(*) AS c FROM products
GROUP BY `productName`, brandid, origin HAVING c > 1) dup ON m.`productName` = dup.`productName` AND m.brandid = dup.brandid AND m.origin = dup.origin
LEFT JOIN tags AS t ON t.productid = m.id
LEFT JOIN groups AS g ON g.productid = m.id
GROUP BY m.`productName`,m.brandid,m.origin
ORDER BY m.`productName`,m.brandid,m.origin
EDIT : You can use this query:
SELECT tt.*
FROM(
SELECT m.*,GROUP_CONCAT(DISTINCT t.tagid ORDER BY t.tagid ASC) AS alltags,GROUP_CONCAT(DISTINCT g.groupid ORDER BY g.groupid ASC) AS groups
FROM `products` m
LEFT JOIN tags AS t ON t.productid = m.id
LEFT JOIN groups AS g ON g.productid = m.id
GROUP BY m.id) tt
INNER JOIN
(SELECT productName,brandid,origin,alltags,groups
FROM
(SELECT m.*,GROUP_CONCAT(DISTINCT t.tagid ORDER BY t.tagid ASC) AS alltags,GROUP_CONCAT(DISTINCT g.groupid ORDER BY g.groupid ASC) AS groups
FROM `products` m
LEFT JOIN tags AS t ON t.productid = m.id
LEFT JOIN groups AS g ON g.productid = m.id
GROUP BY m.id) s
GROUP BY productName,brandid,origin,alltags,groups
HAVING COUNT(*) > 1) ss
ON(tt.productName = ss.productName and tt.brandid = ss.brandid and tt.origin = ss.origin
and tt.alltags = ss.alltags and tt.groups = ss.groups)
I have database with following data structure with sample data. Each company have multiple members. The relationship is in the company_member table. Please note only required fields I have given below.
company
id title
1 company-1
2 company-2
company_member
companyid memberid
1 1
1 2
1 3
2 4
2 5
2 6
member
id firstname member_type_id
1 Name-1 2
2 Name-2 3
3 Name-3 3
4 Name-4 3
5 Name-5 2
6 Name-6 1
member_type
id user_level
1 0
2 1
3 2
I want list of unique companies with one member from each. But the member should be the lowest user_level within the company. i.e, following result should come;
result
companyid company_title memberid member_name user_level
1 company-1 1 Name-1 1
2 company-2 6 Name-6 0
I want to know how to get one member with lowest user level among the same company.
This is a bit complicated one, however this is one way of doing it using not exists, for bigger tables its wise to use not exits since using pivot tables it will not use index.
select
c.id,
c.title,
m.id as member_id,
m.firstname,
mt.user_level
from company_member_map cmp
join company c on c.id = cmp.companyid
join member m on m.id = cmp.memberid
join member_type mt on mt.id = m.member_type_id
where not exists
(
select 1 from company_member_map t1
join member t2 on t2.id = t1.memberid
join member_type t3 on t3.id = t2.member_type_id
where
t1.companyid = cmp.companyid
and t3.user_level < mt.user_level
)
DEMO
Finally I found solution:
select c.id companyid, c.title company_title, m.firstname member_name, mt.user_level
from company c
inner join company_member cm on cm.companyid = c.id
inner join member m on m.id = cm.memberid
inner join member_type mt on mt.id = m.member_type_id
inner join
(select c1.id companyid, mt1.user_level
from company c1
join company_member cm1 on cm1.companyid = c1.id
join member m1 on m1.id = cm1.memberid
join member_type mt1 on mt1.id = m1.member_type_id
group by c1.id,m1.id
order by user_level asc
) sq on c.id = sq.companyid and sq.user_level = mt.user_level
group by c.id;
Correct this, if anyone have better solution or simplified solution.
Check this SQL Fiddle