I have a query that joins four table but I can't understand why its taking about a minute. I know the date between is more than 3 years. I don't really know what to do and how to optimize this query for better performance. Can someone give me suggestion on what to do? Will attached the query and the explain of the query.
SELECT
`purchase_order`.`id`,
`customer`.`name` AS customer_name,
`purchase_order`.`po_date`,
`purchase_order`.`po_number`,
`purchase_order`.`customer_id` AS customer_id ,
`customer`.`name` AS customer_name,
`purchase_order`.`status` AS po_status,
`purchase_order_items`.`product_id`,
`purchase_order_items`.`po_item_name`,
`product`.`weight` as product_weight,
`product`.`pending` as product_pending,
`product`.`company_owner` as company_owner,
`purchase_order_items`.`uom`,
`purchase_order_items`.`po_item_type`,
`purchase_order_items`.`order_sequence`,
`purchase_order_items`.`pending_balance`,
`purchase_order_items`.`quantity`,
`purchase_order_items`.`notes`,
`purchase_order_items`.`status` AS po_item_status,
`purchase_order_items`.`id` AS po_item_id
FROM purchase_order
INNER JOIN customer ON `customer`.`id` = `purchase_order`.`customer_id`
INNER JOIN purchase_order_items ON `purchase_order_items`.`po_id` = `purchase_order`.`id`
INNER JOIN product ON `purchase_order_items`.`product_id` = `product`.`id` WHERE
`purchase_order_items`.`product_id` = '121' AND
`purchase_order`.`po_date`
BETWEEN '2016-01-01' AND '2019-02-28' AND
`purchase_order_items`.`status` IN('Pending','Incomplete')
ORDER BY `purchase_order`.`po_date` DESC LIMIT 0, 20
Im also not sure what to do about the explain and Im still trying to understand why its like this and how can i optimize the query. I hope someone could help me on this.
It looks like your query is good. Rewritten using alias names for tables for shorter yet still readable "po" vs "purchase_order", "poi" vs "purchase_order_items"
Your purchase_order table should have an index at a minimum on date THEN id. So your WHERE clause can optimize based on the date, but also have the po.id as a basis to JOIN to your purchase order items table by id.
Next, the purchase order items, would help with an index on ( po_id, product_id, status ) to help optimize that join criteria.
Purchase_Order index (po_date, id )
Purchase_Order_Items index ( po_id, product_id, status )
SELECT
po.id,
c.name AS customer_name,
po.po_date,
po.po_number,
po.customer_id AS customer_id ,
c.name AS customer_name,
po.status AS po_status,
poi.product_id,
poi.po_item_name,
p.weight as product_weight,
p.pending as product_pending,
p.company_owner as company_owner,
poi.uom,
poi.po_item_type,
poi.order_sequence,
poi.pending_balance,
poi.quantity,
poi.notes,
poi.status AS po_item_status,
poi.id AS po_item_id
FROM
purchase_order po
INNER JOIN customer c
ON po.customer_id = c.id
INNER JOIN purchase_order_items poi
ON po.id = poi.po_id
AND poi.product_id = '121'
AND poi.status IN ('Pending','Incomplete')
INNER JOIN product p
ON poi.product_id = p.id
WHERE
po.po_date BETWEEN '2016-01-01' AND '2019-02-28'
ORDER BY
po.po_date DESC
LIMIT
0, 20
If that doesn't help, Your order of tables appears ok but the engine might be trying to think too much for you. You can try adding additional keyword "STRAIGHT_JOIN" (specific to mysql).
select STRAIGHT_JOIN ( ... rest of query )
This tells mySQL to run the query in the table order I gave you.
Related
I have 2 table that join together ( Orders and Order_item)
When I use join I get duplicates records then I eliminate them with DISTINCT, but when I want to get sum the the shipping_price DISTINCT not works because it just eliminate the same price value and I all my prices are same I get 1 not sum !
If I don't use DISTINCT , I get rows per each items in order
SELECT sum(DISTINCT shipping_price) FROM `product_order`
INNER JOIN `product_address`
ON `product_order`.`address_id` = `product_address`.`id`
INNER JOIN `product_item`
ON `product_order`.`id` = `product_item`.`order_id`
WHERE (`status`<>2)
AND (`company_id`=1968)
AND DATE(date)='2021-08-31'
ORDER BY `product_order`.`id` DESC
how to get sum of shipping price correctly ..
data in order_table is
id shipping_price status
100
200
200
100
sum = 600 , but how to get it, if I did't DISTINCT, I get more than one row per order_items row counts that join tho order..
Use a subquery to get your shipping prices. Something like this:
SELECT product_order.id,
SUM(product_item.price * product_item.quantity) shipping_price
FROM product_order
JOIN product_item ON product_order.id = product_item.order_id
GROUP BY product_order.id
The trick here is to get a subquery that delivers exactly one row per order, with the summed-up shipping priced in it. Do your SUM() ... GROUP BY ...` in the subquery. That way you'll avoid any duplication of items.
TEST THIS before you proceed to make sure it works: giving each order id and its shipping price.
Then use it as if it were a table, JOINing it to the rest.
SELECT total.shipping_price,
product_order.id,
product_address.*
FROM product_order
JOIN product_address
ON product_order.id = product_address.order_id
JOIN (
SELECT product_order.id,
SUM(product_item.price * product_item.quantity) shipping_price
FROM product_order
JOIN product_item ON product_order.id = product_item.order_id
GROUP BY product_order.id
) total ON product_order.id = total.id
ORDER BY product_order.id = total.id
Use a subquery instead of a join.
SELECT (
select sum(shipping_price)
FROM product_item
WHERE po.`id` = `product_item`.`order_id`
) as shipping_price
FROM `product_order` po
INNER JOIN `product_address`
ON `product_order`.`address_id` = `product_address`.`id`
WHERE (`status`<>2)
AND (`company_id`=1968)
AND DATE(date)='2021-08-31'
ORDER BY `product_order`.`id` DESC
I have a mysql query to join four tables and I thought that it was just best to join tables but now that mysql data is getting bigger the query seems to cause the application to stop execution.
SELECT
`purchase_order`.`id`,
`purchase_order`.`po_date` AS po_date,
`purchase_order`.`po_number`,
`purchase_order`.`customer_id` AS customer_id ,
`customer`.`name` AS customer_name,
`purchase_order`.`status` AS po_status,
`purchase_order_items`.`product_id`,
`purchase_order_items`.`po_item_name`,
`product`.`weight` as product_weight,
`product`.`pending` as product_pending,
`product`.`company_owner` as company_owner,
`purchase_order_items`.`uom`,
`purchase_order_items`.`po_item_type`,
`purchase_order_items`.`order_sequence`,
`purchase_order_items`.`pending_balance`,
`purchase_order_items`.`quantity`,
`purchase_order_items`.`notes`,
`purchase_order_items`.`status` AS po_item_status,
`purchase_order_items`.`id` AS po_item_id
FROM `purchase_order`
INNER JOIN customer ON `customer`.`id` = `purchase_order`.`customer_id`
INNER JOIN purchase_order_items ON `purchase_order_items`.`po_id` = `purchase_order`.`id`
INNER JOIN product ON `purchase_order_items`.`product_id` = `product`.`id`
GROUP BY id ORDER BY `purchase_order`.`po_date` DESC LIMIT 0, 20
my problem really is the query that takes a lot of time to finish. Is there a way to speed this query or to change this query for faster retrieval of the data?
heres the EXPLAIN EXTENED as requested in the comments.
Thanks in advance, I really hope this is the right channel for me to ask. If not please let me know.
Will this give you the correct list of ids?
SELECT id
FROM purchase_order
ORDER BY`po_date` DESC
LIMIT 0, 20
If so, then start with that before launching into the JOIN. You can also (I think) get rid of the GROUP BY that is causing an "explode-implode" of rows.
SELECT ...
FROM ( SELECT id ... (as above) ...) AS ids
JOIN purchase_order po ON po.id = ids.id
JOIN ... (the other tables)
GROUP BY ... -- (this may be problematic, especially with the LIMIT)
ORDER BY po.po_date DESC -- yes, this needs repeating
-- no LIMIT
Something like this
SELECT
`purchase_order`.`id`,
`purchase_order`.`po_date` AS po_date,
`purchase_order`.`po_number`,
`purchase_order`.`customer_id` AS customer_id ,
`customer`.`name` AS customer_name,
`purchase_order`.`status` AS po_status,
`purchase_order_items`.`product_id`,
`purchase_order_items`.`po_item_name`,
`product`.`weight` as product_weight,
`product`.`pending` as product_pending,
`product`.`company_owner` as company_owner,
`purchase_order_items`.`uom`,
`purchase_order_items`.`po_item_type`,
`purchase_order_items`.`order_sequence`,
`purchase_order_items`.`pending_balance`,
`purchase_order_items`.`quantity`,
`purchase_order_items`.`notes`,
`purchase_order_items`.`status` AS po_item_status,
`purchase_order_items`.`id` AS po_item_id
FROM (SELECT id, po_date, po_number, customer_id, status
FROM purchase_order
ORDER BY `po_date` DESC
LIMIT 0, 5) as purchase_order
INNER JOIN customer ON `customer`.`id` = `purchase_order`.`customer_id`
INNER JOIN purchase_order_items
ON `purchase_order_items`.`po_id` = `purchase_order`.`id`
INNER JOIN product ON `purchase_order_items`.`product_id` = `product`.`id`
GROUP BY purchase_order.id DESC
LIMIT 0, 5
You need to be sure that purchase_order.po_date and all id column are indexed. You can check it with below query.
SHOW INDEX FROM yourtable;
Since you mentioned that data is getting bigger. I would suggest doing sharding and then you can parallelize multiple queries. Please refer to the following article
Parallel Query for MySQL with Shard-Query
First, I cleaned up readability a bit. You don't need tick marks around every table.column reference. Also, for short-hand, using aliases works well. Ex: "po" instead of "purchase_order", "poi" instead of "purchase_order_items". The only time I would use tick marks is around reserved words that might cause a problem.
Second, you don't have any aggregations (sum, min, max, count, avg, etc.) in your query so you should be able to strip the GROUP BY clause.
As for indexes, I would have to assume you have an index on your reference tables on their respective "id" key columns.
For your Purchase Order table, I would have an index on that based on the "po_date" in the first index field position in case you already had an index using it. Since your Order by is on that, let the engine jump directly to those dated records first and you have your descending order resolved.
SELECT
po.id,
po.po_date,
po.po_number,
po.customer_id,
c.`name` AS customer_name,
po.`status` AS po_status,
poi.product_id,
poi.po_item_name,
p.weight as product_weight,
p.pending as product_pending,
p.company_owner,
poi.uom,
poi.po_item_type,
poi.order_sequence,
poi.pending_balance,
poi.quantity,
poi.notes,
poi.`status` AS po_item_status,
poi.id AS po_item_id
FROM
purchase_order po
INNER JOIN customer c
ON po.customer_id = c.id
INNER JOIN purchase_order_items poi
ON po.id = poi.po_id
INNER JOIN product p
ON poi.product_id = p.id
ORDER BY
po.po_date DESC
LIMIT
0, 20
This almost seems like a scope issue- the select statement in the subquery doesn't recognize table 'candidate':
SELECT
candidate.id AS id,
candidate.image AS image,
candidate.name AS name,
candidate.party AS party,
player.order AS player_order,
c_pcts.pct AS pct
FROM `candidate`
INNER JOIN players player ON player.candidate_id = candidate.id
INNER JOIN lineups lineup ON player.lineup_id = lineup.id
INNER JOIN (
SELECT
pct
FROM candidate_pcts p
INNER JOIN weekly_game game ON p.weekly_game_id = (
SELECT id FROM weekly_game ORDER BY date DESC LIMIT 1
) WHERE p.candidate_id = candidate.id
) c_pcts
WHERE lineup.id = '31'
ORDER BY player.order ASC
gives the error: "Unknown column 'candidate.id' in 'where clause'." If instead of "FROM candidate_pcts p" I put
FROM candidate_pcts p, candidate c
then it doesn't see 'p.weekly_game_id' ...huh?
Seems like I need to identify the 'candidate' table for the subquery somehow but everything I'm trying leads me only further astray. And I have tried a mess of things: order of the tables, explicitly identifying them everywhere i could think of, backticks. I should note that the nested subquery works like a charm. Here it is again:
SELECT
pct
FROM `candidate_pcts`
INNER JOIN weekly_game game ON candidate_pcts.weekly_game_id = (
SELECT id FROM weekly_game ORDER BY date DESC LIMIT 1
) WHERE candidate_pcts.candidate_id = '5'
with a hardcoded an id value there, of course. I can supply database structure if needed here, but this is long already. The 'weekly_game' table is simply a set of scores for each candidate each week and we only want the most recent week's score, thus the 'ORDER BY date DESC LIMIT 1' clause.
Thanks very much for your time.
Tables:
table candidate: {id, image, name, party}
table candidate_pcts: {id, candidate_id, pct, weekly_game_id}
table lineups: {id, date, user_id}
table players: {id,candidate_id,lineup_id,order}
table weekly_game: {id,date}
You are basically on the right track around the problem. In essence the nested sub-select does not know about candidate.id. It you break apart the query and just look at the sub-select in question:
SELECT
pct
FROM candidate_pcts p
INNER JOIN weekly_game game ON p.weekly_game_id = (
SELECT id FROM weekly_game ORDER BY date DESC LIMIT 1
) WHERE p.candidate_id = candidate.id
You can see there is NO reference whatsoever in that query to the candidate table other than in your where clause, thus this is an unknown column.
Since a subselect is, in essence, made before the outer select that references it, the subselect must be a standalone, executable query.
Thanks to all, especially Mike for that excellent explanation. What I did was restructured the query like so:
SELECT
candidate.id AS id,
candidate.image AS image,
candidate.name AS name,
candidate.party AS party,
player.order AS player_order,
pcts.pct AS pct
FROM `candidate`
INNER JOIN players player ON player.candidate_id = candidate.id
INNER JOIN lineups lineup ON player.lineup_id = lineup.id
LEFT JOIN (
SELECT
p.candidate_id AS pct_id, pct AS pct
FROM candidate_pcts p
INNER JOIN weekly_game game ON p.weekly_game_id = (
SELECT id FROM weekly_game ORDER BY date DESC LIMIT 1
)
) pcts
ON pct_id = candidate.id
WHERE lineup.id = '$lineup_id'
ORDER BY player.order ASC
I would like to know how I can write a SQL Script so a within a group of individuals initially selected:
SELECT [RECORDS].[CONSTITUENT_ID]
,[RECORDS].[FIRST_NAME]
,[RECORDS].[LAST_NAME]
,[DATEADDED]
,[DTE]
,[Amount]
,[REF]
,[TYPE]
FROM [re7].[dbo].[GIFT]
INNER JOIN [re7].[dbo].[RECORDS]
ON GIFT.CONSTIT_ID LIKE RECORDS.ID
WHERE ([DTE] BETWEEN '2/7/2015' AND '2/8/2015')
ORDER BY [DATEADDED] DESC
select only individuals who are "First Time Donors" (or someone who only has one gift in [re7].[dbo].[GIFT].
[RECORDS] is a table of all the constituents.
[GIFT] is a table of all recorded Gifts.
The output of the above Query, is just a table with:
CONSTITUENT_ID, FIRST_NAME, LAST_NAME, DATEADDED, DTE, Amount, REF, TYPE
I pretty much want to see the same output format, but I would like the query to select only CONSTITUENT_ID who only have 1 GIFT (by their Record ID) in [re7].[dbo].[GIFT].
I apologize for the lack of data to show. I wish I could describe better....
SELECT [RECORDS].[CONSTITUENT_ID]
,[RECORDS].[FIRST_NAME]
,[RECORDS].[LAST_NAME]
,[DATEADDED]
,[DTE]
,[Amount]
,[REF]
,[TYPE]
FROM [re7].[dbo].[GIFT]
INNER JOIN [re7].[dbo].[RECORDS]
ON GIFT.CONSTIT_ID LIKE RECORDS.ID
WHERE ([DTE] BETWEEN '2/7/2015' AND '2/8/2015')
AND GIFT.CONSTIT_ID IN (
SELECT CONSTIT_ID FROM re7.dbo.Gift GROUP BY CONSTIT_ID HAVING COUNT(*) = 1
) /* another option is to add a subquery to the query you already had */
ORDER BY [DATEADDED] DESC
This solution simply selects all the constituents who have made only one donation and then joins to that, thereby limiting the result set.
SELECT
r.[CONSTITUENT_ID]
,r.[FIRST_NAME]
,r.[LAST_NAME]
,[DATEADDED]
,[DTE]
,[Amount]
,[REF]
,[TYPE]
FROM
(select [CONSTIT_ID] from [re7].[dbo].[GIFT] group by [CONSTIT_ID] having count([CONSTIT_ID]) = 1) g1
inner join [re7].[dbo].[GIFT] g
on g.[CONSTIT_ID] = g1.[CONSTIT_ID]
INNER JOIN [re7].[dbo].[RECORDS] r
ON g.CONSTIT_ID LIKE r.RECORDS.ID
WHERE ([DTE] BETWEEN '2/7/2015' AND '2/8/2015')
ORDER BY [DATEADDED] DESC
I'm running the following query to get the open positions on a portfolio:
SELECT SUM(trades.quantity) as total_quantity, SUM(trades.price) as total_cost, SUM(trades.price)/SUM(trades.quantity) as cost_per_share,
trades.ticker, tickers.code
FROM (trades)
LEFT JOIN tickers
ON trades.ticker = tickers.id
GROUP BY tickers.code
HAVING total_quantity > 0
ORDER BY tickers.code
I'd like to add an extra column to show the weightening of a position, i.e.:
total_cost/SUM(total_cost) -- Dividing any given position cost by the total cost of the portfolio
Since aliases cannot be used in calculations, I thought I'd need to use a sub-query. I've tried a few things but couldn't make it to work.
Can anyone shed some light on this? Is sub-query the way to go? Is there any other better way to do it?
Not sure on your query (you appear to be doing a GROUP BY on a field from a LEFT JOINed table, which could be null for non found matching rows), but maybe cross join to a subselect to get the total of all prices
SELECT total_quantity, total_cost, cost_per_share, trades.ticker, tickers.code, total_cost/total_of_prices
FROM
(
SELECT SUM(trades.quantity) as total_quantity, SUM(trades.price) as total_cost, SUM(trades.price)/SUM(trades.quantity) as cost_per_share,
trades.ticker, tickers.code
FROM trades
LEFT JOIN tickers
ON trades.ticker = tickers.id
GROUP BY tickers.code
HAVING total_quantity > 0
) Sub1
CROSS JOIN
(
SELECT SUM(price) as total_of_prices
FROM trades
WHERE quantity > 0
) Sub2
ORDER BY tickers.code