How can I optimize my sql code? - mysql

I have following tables
contacts
contact_id | contact_slug | contact_first_name | contact_email | contact_date_added | company_id | contact_is_active | contact_subscribed | contact_last_name | contact_company | contact_twitter
contact_campaigns
contact_campaign_id | contact_id | contact_campaign_created | company_id | contact_campaign_sent
bundle_feedback
bundle_feedback_id | bundle_id, contact_id | company_id | bundle_feedback_rating | bundle_feedback_favorite_track_id | bundle_feedback_supporting | campaign_id
bundles
bundle_id | bundle_name | bundle_created | company_id | bundle_is_active
tracks
track_id | company_id | track_title
I wrote this query, but it works slowly, how can I optimize this query to make it faster ?
SELECT SQL_CALC_FOUND_ROWS c.contact_id,
c.contact_first_name,
c.contact_last_name,
c.contact_email,
c.contact_date_added,
c.contact_company,
c.contact_twitter,
concat(c.contact_first_name," ", c.contact_last_name) AS fullname,
c.contact_subscribed,
ifnull(icc.sendCampaignsCount, 0) AS sendCampaignsCount,
ifnull(round((ibf.countfeedbacks/sendCampaignsCount * 100),2), 0) AS percentFeedback,
ifnull(ibf.bundle_feedback_supporting, 0) AS feedbackSupporting
FROM contacts AS c
LEFT JOIN
(SELECT c.contact_id,
count(cc.contact_campaign_id) AS sendCampaignsCount
FROM contacts AS c
LEFT JOIN contact_campaigns AS cc ON cc.contact_id = c.contact_id
WHERE c.company_id = '876'
AND c.contact_is_active = '1'
AND cc.contact_campaign_sent = '1'
GROUP BY c.contact_id) AS icc ON icc.contact_id = c.contact_id
LEFT JOIN
(SELECT bf.contact_id,
count(*) AS countfeedbacks,
bf.bundle_feedback_supporting
FROM bundle_feedback bf
JOIN bundles b
JOIN contacts c
LEFT JOIN tracks t ON bf.bundle_feedback_favorite_track_id = t.track_id
WHERE bf.bundle_id = b.bundle_id
AND bf.contact_id = c.contact_id
AND bf.company_id='876'
GROUP BY bf.contact_id) AS ibf ON ibf.contact_id = c.contact_id
WHERE c.company_id = '876'
AND contact_is_active = '1'
ORDER BY percentFeedback DESC LIMIT 0, 25;

I have done 2 improvements
1) Removed the contacts which is getting joined unnecessarily twice and put the condition at the final where condition.
2) Removed as per SQL_CALC_FOUND_ROWS
Which is fastest? SELECT SQL_CALC_FOUND_ROWS FROM `table`, or SELECT COUNT(*)
SELECT c.contact_id,
c.contact_first_name,
c.contact_last_name,
c.contact_email,
c.contact_date_added,
c.contact_company,
c.contact_twitter,
concat(c.contact_first_name," ", c.contact_last_name) AS fullname,
c.contact_subscribed,
ifnull(icc.sendCampaignsCount, 0) AS sendCampaignsCount,
ifnull(round((ibf.countfeedbacks/sendCampaignsCount * 100),2), 0) AS percentFeedback,
ifnull(ibf.bundle_feedback_supporting, 0) AS feedbackSupporting
FROM contacts AS c
LEFT JOIN
(SELECT cc.contact_id,
count(cc.contact_campaign_id) AS sendCampaignsCount
FROM contact_campaigns
WHERE cc.contact_campaign_sent = '1'
GROUP BY cc.contact_id) AS icc ON icc.contact_id = c.contact_id
LEFT JOIN
(SELECT bf.contact_id,
count(*) AS countfeedbacks,
bf.bundle_feedback_supporting
FROM bundle_feedback bf
JOIN bundles b
LEFT JOIN tracks t ON bf.bundle_feedback_favorite_track_id = t.track_id
WHERE bf.bundle_id = b.bundle_id
GROUP BY bf.contact_id) AS ibf ON ibf.contact_id = c.contact_id
WHERE c.company_id = '876' and c.contact_is_active = '1'

First, you are not identifying any indexes you have to optimize the query. That said, I would ensure you have at least the following composite / covering indexes.
table index
contacts ( company_id, contact_is_active )
contact_campaigns ( contact_id, contact_campaign_sent )
bundle_feedback ( contact_id, bundle_feedback_supporting )
Next, as noted in other answer, unless you really need how many rows qualified, remove the "SQL_CALC_FOUND_ROWS".
In your first left-join (icc), you do a left-join on contact_campaigns (cc), but then throw into your WHERE clause an "AND cc.contact_campaign_sent = '1'" which turns that into an INNER JOIN. At the outer query level, these would result in no matching record and thus NULL for your percentage calculations.
In your second left-join (ibf), you are doing a join to the tracks table, but not utilizing anything from it. Also, you are joining to the bundles table but not using anything from there either -- unless you are getting multiple rows in the bundles and tracks tables which would result in a Cartesian result and possibly overstate your "CountFeedbacks" value. You also do not need the contacts table as you are not doing anything else with it, and the feedback table has the contact ID basis your are querying for. Since that is only grouped by the contact_id, your "bf.bundle_feedback_supporting" is otherwise wasted. If you want counts of feedback, just count from that table per contact ID and remove the rest. (also, the joins should have the "ON" clauses instead of within the WHERE clause for consistency)
Also, for your supporting feedback, the data type and value are unclear, so I implied as a Yes or No and have a SUM() based on how many are supporting. So, a given contact may have 100 records but only 37 are supporting. This gives you 1 record for the contact having BOTH values 100 and 37 respectively and not lost in a group by based on the first entry found for the contact.
I would try to summarize your query to below:
SELECT
c.contact_id,
c.contact_first_name,
c.contact_last_name,
c.contact_email,
c.contact_date_added,
c.contact_company,
c.contact_twitter,
concat(c.contact_first_name," ", c.contact_last_name) AS fullname,
c.contact_subscribed,
ifnull(icc.sendCampaignsCount, 0) AS sendCampaignsCount,
ifnull(round((ibf.countfeedbacks / icc.sendCampaignsCount * 100),2), 0) AS percentFeedback,
ifnull(ibf.SupportCount, 0) AS feedbackSupporting
FROM
contacts AS c
LEFT JOIN
( SELECT
c.contact_id,
count(*) AS sendCampaignsCount
FROM
contacts AS c
JOIN contact_campaigns AS cc
ON c.contact_id = cc.contact_id
AND cc.contact_campaign_sent = '1'
WHERE
c.company_id = '876'
AND c.contact_is_active = '1'
GROUP BY
c.contact_id) AS icc
ON c.contact_id = icc.contact_id
LEFT JOIN
( SELECT
bf.contact_id,
count(*) AS countfeedbacks,
SUM( case when bf.bundle_feedback_supporting = 'Y'
then 1 else 0 end ) as SupportCount
FROM
contacts AS c
JOIN bundle_feedback bf
ON c.contact_id = bf.contact_id
WHERE
c.company_id = '876'
AND c.contact_is_active = '1'
GROUP BY
bf.contact_id) AS ibf
ON c.contact_id = ibf.contact_id
WHERE
c.company_id = '876'
AND c.contact_is_active = '1'
ORDER BY
percentFeedback DESC LIMIT 0, 25;

Related

How can those two SQL statements be combined into one?

I wrote and would like to combine these 2 sql, one is based on results of another. I checked this post, but looks like its not results based. How could I achieve it ?
First sql:
SELECT
`potential`.*,
`customer`.`ID` as 'FID_customer'
FROM
`os_potential` as `potential`,
`os_customer` as `customer`
WHERE `potential`.`FID_author` = :randomID
AND `potential`.`converted` = 1
AND `potential`.`street` = `customer`.`street`
AND `potential`.`zip` = `customer`.`zip`
AND `potential`.`city` = `customer`.`city`;
Second sql:
SELECT
sum(`order`.`price_customer`) as 'Summe'
FROM
`os_order` as `order`,
`RESUTS_FROM_PREVIOUS_SQL_STATEMENT` as `results`
WHERE `order`.`FID_status` = 10
AND `results`.`FID_customer` = `order`.`FID_customer`;
I would like to get everything from first sql + the 'Summe' from second sql.
TABLES
1.Potentials:
+----+------------+-----------+--------+-----+------+
| ID | FID_author | converted | street | zip | city |
+----+------------+-----------+--------+-----+------+
2.Customers:
+----+--------+-----+------+
| ID | street | zip | city |
+----+--------+-----+------+
3.Orders:
+----+--------------+----------------+
| ID | FID_customer | price_customer |
+----+--------------+----------------+
SELECT p.*
, c.ID FID_customer
, o.summe
FROM os_potential p
JOIN os_customer c
ON c.street = p.street
AND c.zip = p.zip
AND c.city = p.city
JOIN
( SELECT FID_customer
, SUM(price_customer) Summe
FROM os_order
WHERE FID_status = 10
GROUP
BY FID_customer
) o
ON o.FID_customer = c.ID
WHERE p.FID_author = :randomID
AND p.converted = 1
;
You would just write a single query like this:
SELECT sum(o.price_customer) as Summe
FROM os_order o JOIN
os_potential p JOIN
os_customer c
ON p.street = c.street AND p.zip = c.zip AND p.city = c.city JOIN
os_order o2
ON o2.FID_customer = c.FID_customer
WHERE p.FID_author = :randomID AND p.converted = 1 AND
o2.FID_status = 10 ;
Notes:
Never use commas in the FROM clause. Always use explicit JOIN syntax with conditions in an ON clause.
Table aliases are easier to follow when they are short. Abbreviations for the table names is commonly used.
Backticks are only necessary when the table/column name needs to be escaped. Yours don't need to be escaped.
If the 1st query return 1 record per customer, then just simply join the 3 tables, keep the sum and use the group by clause:
SELECT
`potential`.*,
`customer`.`ID` as 'FID_customer',
sum(`order`.`price_customer`) as Summe
FROM
`os_potential` as `potential`
INNER JOIN
`os_customer` as `customer`
ON `potential`.`street` = `customer`.`street`
AND `potential`.`zip` = `customer`.`zip`
AND `potential`.`city` = `customer`.`city`
LEFT JOIN
`os_order` as `order`
ON `results`.`FID_customer` = `order`.`FID_customer`
AND `order`.`FID_status` = 10
WHERE `potential`.`FID_author` = :randomID
AND `potential`.`converted` = 1
GROUP BY `customer`.`ID`, <list all fields from potential table>
If the 1st query may return multiple records per customer, then you need to do the summing in a subquery:
SELECT
`potential`.*,
`customer`.`ID` as 'FID_customer',
`order`.Summe
FROM
`os_potential` as `potential`
INNER JOIN
`os_customer` as `customer`
ON `potential`.`street` = `customer`.`street`
AND `potential`.`zip` = `customer`.`zip`
AND `potential`.`city` = `customer`.`city`
LEFT JOIN
(SELECT FID_customer, sum(price_customer) as Summe
FROM `os_order`
WHERE FID_status=10
GROUP BY FID_customer
) as `order`
ON `results`.`FID_customer` = `order`.`FID_customer`
WHERE `potential`.`FID_author` = :randomID
AND `potential`.`converted` = 1
I think you should use a subselect, but be careful with the number of results, it's not the best for performance.
You can do something like this:
SELECT n1, n2, (select count(1) from whatever_table) as n3, n4 from whatever_table
note that the subselect must return just 1 result, in other case you'll have an error

SQL query left join where clause

This is my situation.
I have 3 tables
Orders
- id status deleted
Order Lines
- related_id related_model quantity
Products
- id code price price_purchase
I want to create a list with all products. The amount of times they are purchased and a sum of the gross margin (price - price_purchase). It must only use orders lines with the related model set to 'products'. And secondly it must only pick orders with the status set to 'paid, processing, sent, ready_for_pickup or picked_up' and with the order not deleted.
So this would be the result I want:
id | code | purchases | value
-------------------------------
1 | code1 | 7 | 57,05
2 | code2 | 122 | 254,98
3 | code3 | 0 | 0,00
This is the SQL query I have so far:
SELECT p.id, p.code, IFNULL(SUM(sol.quantity) , 0) as purcahses,
sum((p.price - p.price_purchase) * quantity) as value
FROM products p
LEFT JOIN shop_orders_lines sol ON sol.related_id = p.id
AND sol.related_model = 'products'
LEFT JOIN shop_orders so ON so.id = sol.order_id
WHERE so.status IN ('paid', 'processing', 'sent', 'ready_for_pickup', 'picked_up')
AND so.deleted = 0
GROUP BY p.id
It returns the correct data. But not all problems. That is my problem. I a lot of different methods like sub queries and other methods but can't seem to solve the problem. I know the problem is my LEFT join, but don't know a solution to my problem.
I'm using MySQL Workbench.
Any help is welcome.
Your joins are wrong. You need to identify the order lines to consider separately from and prior to forming the LEFT JOIN with the product details. An inline view could help:
SELECT
p.id,
p.code,
IFNULL(SUM(ordered_item.quantity) , 0) as purchases ,
sum((p.price - p.price_purchase) * ordered_item.quantity) as value
FROM
products p
LEFT JOIN (
SELECT
sol.related_id AS related_id,
sol.quantity AS quantity
FROM
shop_orders_lines sol
INNER JOIN shop_orders so
ON so.id = sol.order_id
WHERE
so.status IN ('paid', 'processing', 'sent', 'ready_for_pickup', 'picked_up')
AND so.deleted = 0
AND sol.related_model = 'products'
) ordered_item
ON ordered_item.related_id = p.id
GROUP BY p.id
Move outer table conditions from WHERE to ON, otherwise the OUTER JOIN works like a regular INNER JOIN:
SELECT p.id, p.code, IFNULL(SUM(sol.quantity) , 0) as purcahses,
sum((p.price - p.price_purchase) * quantity) as value
FROM products p
LEFT JOIN shop_orders_lines sol ON sol.related_id = p.id
AND sol.related_model = 'products'
LEFT JOIN shop_orders so ON so.id = sol.order_id AND
so.status IN ('paid', 'processing', 'sent', 'ready_for_pickup', 'picked_up')
AND so.deleted = 0
GROUP BY p.id
Is p.id the whole primary key for that table? If not, you need to find out how to treat p.code. (Either list in GROUP BY, or use as argument to aggregate function.)
Another try:
SELECT p.id, p.code, IFNULL(SUM(sol.quantity) , 0) as purcahses,
sum((p.price - p.price_purchase) * quantity) as value
FROM products p
JOIN shop_orders_lines sol ON sol.related_id = p.id
AND sol.related_model = 'products'
WHERE EXISTS (select 1 from shop_orders so
where so.id = sol.order_id
AND so.status IN ('paid', 'processing', 'sent', 'ready_for_pickup', 'picked_up')
AND so.deleted = 0)
GROUP BY p.id

MYSQL - Union SELECT & GROUP BY Not working

I have 3 table, log, member, also guest, but my log i stored as customer(user)'s id only, which is either their guest_id or member_id. So here's the problem, because they're from different table, I'm not sure how to join & group together their data.
checkout_log table
id user_id checkout_as
--------------------------------------
1 1 member
2 2 guest
members table
id fullname
--------------------------------------
1 member01
2 member02
guests table
id fullname
--------------------------------------
1 guest01
2 guest02
What I wanted to Achieve - Result
id user_id fullname checkout_as
----------------------------------------------
1 1 member01 member
2 2 guest02 guest
Had tried following sql statement with UNION ALL, or GROUP BY , but had no luck.
SELECT * FROM
(
SELECT checkout_log.id,checkout_log.user_id,guests.fullname,guests.email,checkout_log.checkout_as
FROM checkout_log
LEFT JOIN checkout_product ON checkout_product.checkout_log_id = checkout_log.id
LEFT JOIN guests ON checkout_log.user_id = guests.id
UNION ALL
SELECT checkout_log.id,checkout_log.user_id,members.fullname,members.email,checkout_log.checkout_as
FROM checkout_log
LEFT JOIN checkout_product ON checkout_product.checkout_log_id = checkout_log.id
LEFT JOIN members ON checkout_log.user_id = members.id
) derivedTable
GROUP BY id
Try doing this with joins instead of union
select cl.id, cl.user_id,
coalesce(m.fullname, g.fullname) as fullname,
cl.checkout_as
from checkout_log cl left join
members m
on cl.user_id = m.id and cl.checkout_as = 'member' left join
guests g
on cl.user_id = g.id and cl.checkout_as = 'guest';

MySQL join 3 calculation results

I have 3 different MySQL calculations, which I'd like to join. I need to be able to show lines where a sum may not exist for some column, or some invoice might not have a corporation ID.
I'm trying to get something like:
passport_amount | invoice_amount | balance_amount | corporation_id
------------------------------------------------------------------
345 | 2345 | 56 | 56
So that I can work on these values in my application code by iterating the list once and not fetching data from the database three times, and then iterating three times to combine the values.
SELECT
sum(passports.amount) AS passports_amount,
companies.corporation_id
FROM
passports
INNER JOIN
employees ON ( passports.employee_id = employees.id )
INNER JOIN
companies ON ( employees.company_id = companies.id )
WHERE
((((passports.pass_type IN ('sport','culture','both'))
AND
(MONTH(passports.valid_from) >= 1
AND MONTH(passports.valid_from) <= 9
AND YEAR(passports.valid_from) = year(now())))
AND (passports.removed = 0
AND passports.valid_from <= date('2014-09-29 11:55:26')))
AND (companies.removed = 0)
AND companies.corporation_id IS NOT NULL)
GROUP BY
companies.corporation_id;
SELECT
sum(invoices.amount) AS invoices_amount,
invoices.corporation_id
FROM
invoices
WHERE
((((YEAR(sent_at) = 2014)
AND (invoices.product_type_id IN (2,3,4)))
AND
(invoices.removed = 0
AND invoices.activated = 1))
AND invoices.corporation_id IS NOT NULL)
GROUP BY
invoices.corporation_id;
SELECT
amount AS balance_amount,
business_id AS corporation_id
FROM
invoice_balances
WHERE
business_type = 'Corporation';
You can combine queries for balance and invoice amount using sub select in left join but this will look odd and can be expensive in terms of performance
SELECT
SUM(p.amount) AS passports_amount,
ii.invoices_amount,
b.balance_amount,
c.corporation_id
FROM
passports p
INNER JOIN employees e ON ( p.e = e.id )
INNER JOIN companies c ON ( e.company_id = c.id )
/* Added left join for balance using subselect*/
LEFT JOIN (
SELECT amount AS balance_amount, business_id AS corporation_id
FROM invoice_balances
WHERE business_type = 'Corporation'
) b ON (c.corporation_id = b.corporation_id)
/* Added left join for invoices_amount using subselect*/
LEFT JOIN(
SELECT SUM(i.amount) AS invoices_amount,
i.corporation_id
FROM
invoices i
WHERE
((((YEAR(sent_at) = 2014)
AND (i.product_type_id IN (2,3,4)))
AND (i.removed = 0 AND i.activated = 1)
)
AND i.corporation_id IS NOT NULL)
GROUP BY i.corporation_id
) ii ON(c.corporation_id = ii.corporation_id)
/* end of joins */
WHERE
((((p.pass_type IN ('sport','culture','both'))
AND
(MONTH(p.valid_from) >= 1
AND MONTH(p.valid_from) <= 9
AND YEAR(p.valid_from) = YEAR(NOW())))
AND (p.removed = 0
AND p.valid_from <= DATE('2014-09-29 11:55:26')))
AND (c.removed = 0)
AND c.corporation_id IS NOT NULL)
GROUP BY c.corporation_id;

sql nested join inner and left join

Hi I want to filter logs using MySQL with a list of trackings.
Every log belongs to a server,
Every tracking belongs to a server and have 0..N patterns
Every pattern belongs to a tracking
I have 3 tables :
logs : | id | ip | url | server_id | ...
tracking : | id | server_id | name | other fields...
pattern : | id | tracking_id | pattern |
I want to count logs that match tracking for a specific server my problem is that my query mix up tracking that have pattern and those that don't.
SQL Fiddle : http://sqlfiddle.com/#!2/f11b1/2
SELECT COUNT(DISTINCT logs.ip), tr.name
FROM `logs`
INNER JOIN `trackings` as tr ON
( tr.server_id = logs.server_id )
AND -- OTHER conditions between log and tracking
LEFT JOIN `patterns` as pt ON
( pt.tracking_id = tr.id )
AND (logs.url LIKE pt.pattern )
GROUP BY tr.id
My problem is on the second join, if I use INNER JOIN patterns as pt ON I get correct results but only on trackings that have some patterns,
If I use LEFT JOIN patterns as pt ON I get all tracking but with a false count (I get the result of SELECT COUNT(DISTINCT logs.ip) FROM logs )
EDIT
I Can get the correct result with a field in tracking that indicates if the tracking has patterns and a UNION :
(
SELECT COUNT(DISTINCT lg.ip), tr.name
FROM `logs` as lg
INNER JOIN `trackings` as tr ON
( tr.server_id = lg.server_id )
AND (tr.hasPatterns = 1)
AND -- Other conditions
INNER JOIN `patterns` as pt ON
( pt.tracking_id = tr.id )
AND (lg.url LIKE pt.pattern )
WHERE
GROUP BY tr.id
)
UNION
(
SELECT COUNT(DISTINCT lg.ip), tr.name
FROM `logs` as lg
INNER JOIN `trackings` as tr ON
( tr.server_id = lg.server_id )
AND (tr.hasPatterns = 0)
WHERE
GROUP BY tr.id, lg.date
)
But I guess there is a way to do that without using Union...
You can put a conditional inside count, so I think the following does what you want:
SELECT COUNT(DISTINCT (case when tr.hasPatterns = 1 and pt.tracking_id is not null
then lg.ip
when tr.hasPatterns = 0
then lg.ip
end)), tr.name
FROM `logs` as lg
INNER JOIN `trackings` as tr ON
( tr.server_id = lg.server_id )
AND -- Other conditions
LEFT JOIN `patterns` as pt ON
( pt.tracking_id = tr.id )
AND (lg.url LIKE pt.pattern )
WHERE
GROUP BY tr.id
EDIT:
This is returning what you want:
SELECT COUNT(DISTINCT (case when tr.size = 0 and pt.tracking_id is not null
then lg.ip
when tr.size > 0 and lg.size > tr.size
then lg.ip
end)), tr.name
FROM `logs` as lg
INNER JOIN `trackings` as tr ON
( tr.server_id = lg.server_id )
LEFT JOIN `patterns` as pt ON
( pt.tracking_id = tr.id )
AND (lg.url LIKE pt.pattern )
GROUP BY tr.id;
Your SQL Fiddle has the additional condition lg.size > tr.size which is not in the original question.