Sum of rows with join - mysql

This is the current table layout.
There are 3 legs
Each leg has 2 points, where is_start = 1 is the start of the leg, and is_start is the end of the leg.
When the user check in at a point, a entry in points_user are created.
In this application you have multiple legs which has 2 points where one marks the start of the leg, where the other marks the end of the leg. So the sum of User's (with id = 2) Leg (with id= 1) is points_users.created where points_users.leg_id = 1 and points_users.user_id = 2 and points_users.is_start = 0 minus points_users where is_start = 1 (and the other parameters stay the same). And that's for just one leg.
What I would like is to sum all the time differences for each leg, we get the data like this:
| User.id | User.name | total_time |
| 1 | John | 129934 |
Anyone know how I can join these tables and sum it up grouped by user?
(No, this is not homework)
As far as I got:
SELECT
( `end_time` - `start_time` ) AS `diff`
FROM
(
SELECT SUM(UNIX_TIMESTAMP(`p1`.`created`)) AS `start_time`
FROM `points_users` AS `pu1`
LEFT JOIN `points` AS `p1` ON `pu1`.`point_id` = `p1`.`id`
WHERE `p1`.`is_start` = 1
) AS `start_time`,
(
SELECT SUM(UNIX_TIMESTAMP(`pu2`.`created`)) AS `end_time`
FROM `points_users` AS `pu2`
LEFT JOIN `points` AS `p2` ON `pu2`.`point_id` = `p2`.`id`
WHERE `p2`.`is_start` = 0
) AS `end_time`

Try this:
select users.user_id,
users.user_name,
SUM(timeDuration) totalTime
from users
join (
select
pStart.User_id,
pStart.leg_id,
(pEnd.created - pStart.created) timeDuration
from (select pu.user_id, pu.leg_id, pu.created
from points_users pu
join points p on pu.id = p.point_id and pu.leg_id = p.leg_id
where p.is_start = 1 ) pStart
join (select pu.user_id, pu.leg_id, pu.created
from points_users pu
join points p on pu.id = p.point_id and pu.leg_id = p.leg_id
where p.is_start = 0 ) pEnd
on pStart.user_id = pEnd.user_id
and pStart.leg_id = pEnd.leg_id
) tt
on users.user_id = tt.user_id
group by users.user_id, users.user_name
Subquery gets the time duration for each user/leg, and main query then sums them for all the legs of each user.

EDIT: Added the points table now that I can see your attempt at a query.
The simplest way is to join points_users to itself:
select leg_start.user_id, sum(leg_end.created - leg_start.created)
from points_users leg_start
join points_users leg_end on leg_start.user_id = leg_end.user_id
and leg_start.leg_id = leg_end.leg_id
join points point_start on leg_start.point_id = point_start.id
join points point_end on leg_end.point_id = point_end.id
where point_start.is_start = 1 and point_end.is_start = 0
group by leg_start.user_id
Some people prefer to put those is_start filters in the join condition, but since it's an inner join that's mainly just a point of style. If it were an outer join, then moving them from the WHERE to the JOIN could have an effect on the results.

Related

how optimize prestashop category get product for random

this is prestashop 1.7 version category get product query. if use random, it is very slow, how optimize it?
SELECT
cp.id_category,
p.*,
product_shop.*,
stock.out_of_stock,
IFNULL( stock.quantity, 0 ) AS quantity,
IFNULL( product_attribute_shop.id_product_attribute, 0 ) AS id_product_attribute,
product_attribute_shop.minimal_quantity AS product_attribute_minimal_quantity,
pl.`description`,
pl.`description_short`,
pl.`available_now`,
pl.`available_later`,
pl.`link_rewrite`,
pl.`meta_description`,
pl.`meta_keywords`,
pl.`meta_title`,
pl.`name`,
image_shop.`id_image` id_image,
il.`legend` AS legend,
m.`name` AS manufacturer_name,
cl.`name` AS category_default,
DATEDIFF(
product_shop.`date_add`,
DATE_SUB( "2019-11-30 00:00:00", INTERVAL 7 DAY )) > 0 AS new,
product_shop.price AS orderprice
FROM
`ps_category_product` cp
LEFT JOIN `ps_product` p ON p.`id_product` = cp.`id_product`
INNER JOIN ps_product_shop product_shop ON ( product_shop.id_product = p.id_product AND product_shop.id_shop = 1 )
LEFT JOIN `ps_product_attribute_shop` product_attribute_shop ON ( p.`id_product` = product_attribute_shop.`id_product` AND product_attribute_shop.`default_on` = 1 AND product_attribute_shop.id_shop = 1 )
LEFT JOIN ps_stock_available stock ON ( stock.id_product = `p`.id_product AND stock.id_product_attribute = 0 AND stock.id_shop = 1 AND stock.id_shop_group = 0 )
LEFT JOIN `ps_category_lang` cl ON ( product_shop.`id_category_default` = cl.`id_category` AND cl.`id_lang` = 11 AND cl.id_shop = 1 )
LEFT JOIN `ps_product_lang` pl ON ( p.`id_product` = pl.`id_product` AND pl.`id_lang` = 11 AND pl.id_shop = 1 )
LEFT JOIN `ps_image_shop` image_shop ON ( image_shop.`id_product` = p.`id_product` AND image_shop.cover = 1 AND image_shop.id_shop = 1 )
LEFT JOIN `ps_image_lang` il ON ( image_shop.`id_image` = il.`id_image` AND il.`id_lang` = 11 )
LEFT JOIN `ps_manufacturer` m ON m.`id_manufacturer` = p.`id_manufacturer`
WHERE
product_shop.`id_shop` = 1
AND cp.`id_category` = 12
AND product_shop.`active` = 1
AND product_shop.`visibility` IN ( "both", "catalog" )
ORDER BY
RAND()
LIMIT 50
Please provide SHOW CREATE TABLE for each table. Meanwhile, ...
Let's start by optimizing the joins.
LEFT JOIN `ps_product_lang` pl ON ( p.`id_product` = pl.`id_product`
AND pl.`id_lang` = 11
AND pl.id_shop = 1 )
That needs INDEX(id_product, id_lang, id_shop) (The columns may be in any order.)
Don't use LEFT unless you really need to fetch a row from the righthand table as NULLs when it does not exist. In particular,
LEFT JOIN `ps_product` p
is probably getting in the way of optimization.
WHERE product_shop.`id_shop` = 1
AND product_shop.`active` = 1
AND product_shop.`visibility` IN ( "both", "catalog" )
would probably benefit from these indexes
INDEX(id_shop, active, visibility, id_product)
INDEX(id_product, id_shop, active, visibility)
product_category needs
INDEX(id_category, id_product) -- in this order.
In general many-to-many mapping tables need to follow the tips here: http://mysql.rjweb.org/doc.php/index_cookbook_mysql#many_to_many_mapping_table
The query has the "explode-implode" syndrome. This is where it first does a JOINs, collecting a lot of data, then throws away much of it due, in your case, to the LIMIT 10. It can probably be cured by turning the query inside-out. The general ID is to start with a derived table that gets the 10 rows desired, then reaches into the other table for the rest of the desired columns. This "reaching" need happen only 10 times, not however many the JOINs currently require.
SELECT ...
FROM ( SELECT <<primary key columns from cp, p, and product_shop>>
FROM cp
JOIN p ON ...
JOIN product_shop ON ...
ORDER BY RAND()
LIMIT 10 ) AS x
JOIN <<p, product_shop ON their PKs>> -- to get p.*, product_shop.*>>
[LEFT] JOIN << each of the other tables>> -- to get the other tables
You should start by testing the subquery (a "derived" table) to verify that it is noticeably faster than the original query.

Adding a subquery to a join

I've been using the following join, to pull rows of users whom have volunteered for various project positions.
SELECT p.id, up.position_id, title, max_vol, current_datetime, IF(up.id IS NULL, "0", "1") volunteered
FROM positions AS p
LEFT JOIN users_positions AS up
ON p.id = up.position_id
AND up.user_id = 1
AND up.calendar_date = '2016-10-03'
WHERE
p.project_id = 1
AND p.day = 1
...but in a change of functionality, I have to now account for the date of the latest edit to a project. In another query, I solved it like so
SELECT *
FROM positions
WHERE
current_datetime = (SELECT MAX(current_datetime)
FROM positions
WHERE
project_id = 1 AND day = 1)
Which works fine, but now I have to also incorporate the return of rows which match the latest datetime in the left join query.
I just can't seem to wrap my head around it. Any suggestions? Thanks.
Use a sub query, like this:
SELECT
p.id,
up.position_id,
title,
max_vol,
current_datetime,
IF(up.id IS NULL,
"0",
"1") volunteered
FROM
( SELECT
*
FROM
positions
WHERE
current_datetime = (
SELECT
MAX(current_datetime)
FROM
positions
WHERE
project_id = 1
AND day = 1
)
) AS p
LEFT JOIN
users_positions AS up
ON p.id = up.position_id
AND up.user_id = 1
AND up.calendar_date = '2016-10-03'
WHERE
p.project_id = 1
AND p.day = 1

MySQL join 3 calculation results

I have 3 different MySQL calculations, which I'd like to join. I need to be able to show lines where a sum may not exist for some column, or some invoice might not have a corporation ID.
I'm trying to get something like:
passport_amount | invoice_amount | balance_amount | corporation_id
------------------------------------------------------------------
345 | 2345 | 56 | 56
So that I can work on these values in my application code by iterating the list once and not fetching data from the database three times, and then iterating three times to combine the values.
SELECT
sum(passports.amount) AS passports_amount,
companies.corporation_id
FROM
passports
INNER JOIN
employees ON ( passports.employee_id = employees.id )
INNER JOIN
companies ON ( employees.company_id = companies.id )
WHERE
((((passports.pass_type IN ('sport','culture','both'))
AND
(MONTH(passports.valid_from) >= 1
AND MONTH(passports.valid_from) <= 9
AND YEAR(passports.valid_from) = year(now())))
AND (passports.removed = 0
AND passports.valid_from <= date('2014-09-29 11:55:26')))
AND (companies.removed = 0)
AND companies.corporation_id IS NOT NULL)
GROUP BY
companies.corporation_id;
SELECT
sum(invoices.amount) AS invoices_amount,
invoices.corporation_id
FROM
invoices
WHERE
((((YEAR(sent_at) = 2014)
AND (invoices.product_type_id IN (2,3,4)))
AND
(invoices.removed = 0
AND invoices.activated = 1))
AND invoices.corporation_id IS NOT NULL)
GROUP BY
invoices.corporation_id;
SELECT
amount AS balance_amount,
business_id AS corporation_id
FROM
invoice_balances
WHERE
business_type = 'Corporation';
You can combine queries for balance and invoice amount using sub select in left join but this will look odd and can be expensive in terms of performance
SELECT
SUM(p.amount) AS passports_amount,
ii.invoices_amount,
b.balance_amount,
c.corporation_id
FROM
passports p
INNER JOIN employees e ON ( p.e = e.id )
INNER JOIN companies c ON ( e.company_id = c.id )
/* Added left join for balance using subselect*/
LEFT JOIN (
SELECT amount AS balance_amount, business_id AS corporation_id
FROM invoice_balances
WHERE business_type = 'Corporation'
) b ON (c.corporation_id = b.corporation_id)
/* Added left join for invoices_amount using subselect*/
LEFT JOIN(
SELECT SUM(i.amount) AS invoices_amount,
i.corporation_id
FROM
invoices i
WHERE
((((YEAR(sent_at) = 2014)
AND (i.product_type_id IN (2,3,4)))
AND (i.removed = 0 AND i.activated = 1)
)
AND i.corporation_id IS NOT NULL)
GROUP BY i.corporation_id
) ii ON(c.corporation_id = ii.corporation_id)
/* end of joins */
WHERE
((((p.pass_type IN ('sport','culture','both'))
AND
(MONTH(p.valid_from) >= 1
AND MONTH(p.valid_from) <= 9
AND YEAR(p.valid_from) = YEAR(NOW())))
AND (p.removed = 0
AND p.valid_from <= DATE('2014-09-29 11:55:26')))
AND (c.removed = 0)
AND c.corporation_id IS NOT NULL)
GROUP BY c.corporation_id;

sql nested join inner and left join

Hi I want to filter logs using MySQL with a list of trackings.
Every log belongs to a server,
Every tracking belongs to a server and have 0..N patterns
Every pattern belongs to a tracking
I have 3 tables :
logs : | id | ip | url | server_id | ...
tracking : | id | server_id | name | other fields...
pattern : | id | tracking_id | pattern |
I want to count logs that match tracking for a specific server my problem is that my query mix up tracking that have pattern and those that don't.
SQL Fiddle : http://sqlfiddle.com/#!2/f11b1/2
SELECT COUNT(DISTINCT logs.ip), tr.name
FROM `logs`
INNER JOIN `trackings` as tr ON
( tr.server_id = logs.server_id )
AND -- OTHER conditions between log and tracking
LEFT JOIN `patterns` as pt ON
( pt.tracking_id = tr.id )
AND (logs.url LIKE pt.pattern )
GROUP BY tr.id
My problem is on the second join, if I use INNER JOIN patterns as pt ON I get correct results but only on trackings that have some patterns,
If I use LEFT JOIN patterns as pt ON I get all tracking but with a false count (I get the result of SELECT COUNT(DISTINCT logs.ip) FROM logs )
EDIT
I Can get the correct result with a field in tracking that indicates if the tracking has patterns and a UNION :
(
SELECT COUNT(DISTINCT lg.ip), tr.name
FROM `logs` as lg
INNER JOIN `trackings` as tr ON
( tr.server_id = lg.server_id )
AND (tr.hasPatterns = 1)
AND -- Other conditions
INNER JOIN `patterns` as pt ON
( pt.tracking_id = tr.id )
AND (lg.url LIKE pt.pattern )
WHERE
GROUP BY tr.id
)
UNION
(
SELECT COUNT(DISTINCT lg.ip), tr.name
FROM `logs` as lg
INNER JOIN `trackings` as tr ON
( tr.server_id = lg.server_id )
AND (tr.hasPatterns = 0)
WHERE
GROUP BY tr.id, lg.date
)
But I guess there is a way to do that without using Union...
You can put a conditional inside count, so I think the following does what you want:
SELECT COUNT(DISTINCT (case when tr.hasPatterns = 1 and pt.tracking_id is not null
then lg.ip
when tr.hasPatterns = 0
then lg.ip
end)), tr.name
FROM `logs` as lg
INNER JOIN `trackings` as tr ON
( tr.server_id = lg.server_id )
AND -- Other conditions
LEFT JOIN `patterns` as pt ON
( pt.tracking_id = tr.id )
AND (lg.url LIKE pt.pattern )
WHERE
GROUP BY tr.id
EDIT:
This is returning what you want:
SELECT COUNT(DISTINCT (case when tr.size = 0 and pt.tracking_id is not null
then lg.ip
when tr.size > 0 and lg.size > tr.size
then lg.ip
end)), tr.name
FROM `logs` as lg
INNER JOIN `trackings` as tr ON
( tr.server_id = lg.server_id )
LEFT JOIN `patterns` as pt ON
( pt.tracking_id = tr.id )
AND (lg.url LIKE pt.pattern )
GROUP BY tr.id;
Your SQL Fiddle has the additional condition lg.size > tr.size which is not in the original question.

using joins together with aggregates, and retrieving rows when no aggregate exists

The following query on my MySQL tables returns rows from the purchaseorder table that have corresponding entries in the deliveryorder table. How do I construct this query so that I get rows from the purchaseorder table even if no corresponding rows exist in the deliveryorder table? If the users want to see sql table CREATE statements, I can post those, but I'm not posting now as it really makes the question too big.
SELECT
`purchaseorder`.`id` AS `po_id`,
`purchaseorder`.`order_quantity` AS `po_order_quantity`,
`purchaseorder`.`applicable_approved_unit_rate` AS `po_unit_rate`,
`purchaseorder`.`applicable_sales_tax_rate` AS `po_tax_rate`,
`purchaseorder`.`order_date` AS `po_order_date`,
`purchaseorder`.`remarks` AS `po_remarks`,
`purchaseorder`.`is_open` AS `po_is_open`,
`purchaseorder`.`is_active` AS `po_is_active`,
`purchaseorder`.`approved_rate_id` AS `po_app_rate_id`,
`supplier`.`name` AS `sup_name`,
SUM(`deliveryorder`.`quantity`) AS `total_ordered`
FROM `purchaseorder`
LEFT JOIN `deliveryorder` ON (`deliveryorder`.`purchase_order_id` = `purchaseorder`.`id`)
INNER JOIN `approvedrate` ON (`purchaseorder`.`approved_rate_id` = `approvedrate`.`id`)
INNER JOIN `supplier` ON (`approvedrate`.`supplier_id` = `supplier`.`id`)
WHERE (
`purchaseorder`.`is_active` = 1
AND `purchaseorder`.`is_open` = 1
AND `deliveryorder`.`is_active` = 1
AND `approvedrate`.`material_id` = 2
)
HAVING `purchaseorder`.`order_quantity` >= `total_ordered` + 1
You have an aggregating function but no GROUP BY clause, which is wierd, but anyway - something like this? Oops - edited...
SELECT po.id po_id
, po.order_quantity po_order_quantity
, po.applicable_approved_unit_rate po_unit_rate
, po.applicable_sales_tax_rate po_tax_rate
, po.order_date po_order_date
, po.remarks po_remarks
, po.is_open po_is_open
, po.is_active po_is_active
, po.approved_rate_id po_app_rate_id
, s.name sup_name
, SUM(do.quantity) total_ordered
FROM purchaseorder po
LEFT
JOIN deliveryorder do
ON do.purchase_order_id = po.
AND do.is_active = 1
LEFT
JOIN approvedrate ar
ON ar.id = po.approved_rate_id
AND ar.material_id = 2
LEFT
JOIN supplier s
ON s.id = ar.supplier_id
WHERE po.is_active = 1
AND po.is_open = 1
HAVING po.order_quantity >= total_ordered + 1
I couldn't work out how to get the desired results all in one query, but ended up using the following two queries to fulfill my requirements: -
1st query
SELECT
pot.`id` AS `po_id`,
pot.`order_quantity` AS `po_order_quantity`,
pot.`applicable_approved_unit_rate` AS `po_unit_rate`,
pot.`applicable_sales_tax_rate` AS `po_tax_rate`,
pot.`is_open` AS `po_is_open`,
pot.`is_active` AS `po_is_active`,
st.`id` AS `sup_id`,
st.`name` AS `sup_name`,
SUM(dot.`quantity`) AS `total_ordered`
FROM `purchaseorder` pot
INNER JOIN `deliveryorder` dot ON (dot.`purchase_order_id` = pot.`id`)
INNER JOIN `approvedrate` art ON (pot.`approved_rate_id` = art.`id`)
INNER JOIN `supplier` st ON (art.`supplier_id` = st.`id`)
WHERE (
pot.`is_active` = 1
AND pot.`is_open` = 1
AND art.`material_id` = #materialid
AND art.`in_effect` = 1
AND art.`is_active` = 1
AND dot.`is_active` = 1
AND st.`is_active` = 1
)
HAVING pot.`order_quantity` >= `total_ordered` + #materialquantity
2nd query
SELECT
pot.`id` AS `po_id`,
pot.`order_quantity` AS `po_order_quantity`,
pot.`applicable_approved_unit_rate` AS `po_unit_rate`,
pot.`applicable_sales_tax_rate` AS `po_tax_rate`,
pot.`is_open` AS `po_is_open`,
pot.`is_active` AS `po_is_active`,
st.`id` AS `sup_id`,
st.`name` AS `sup_name`,
0 AS `total_ordered`
FROM `purchaseorder` pot
INNER JOIN `approvedrate` art ON (pot.`approved_rate_id` = art.`id`)
INNER JOIN `supplier` st ON (art.`supplier_id` = st.`id`)
WHERE (
pot.`is_active` = 1
AND pot.`is_open` = 1
AND art.`material_id` = #materialid
AND art.`in_effect` = 1
AND art.`is_active` = 1
AND st.`is_active` = 1
AND pot.`order_quantity` >= #materialquantity
AND pot.`id` NOT IN
(
SELECT dot.`purchase_order_id`
FROM `deliveryorder` dot
WHERE dot.is_active = 1
)
)