I convert an old software (that use MS-ACCESS MDB) to mySQL.
I have a query that takes long time to run (actualy I break running after 5 minutes of waiting)
How can I write it?
SELECT pa_ID, pa_PRODUCT_ID, pr_ID,pr_NAME,Sum(pa_KILOS) as IN_KILOS,
(select sum(pl_KILOS) from POLHSH where POLHSH.pl_PRODUCT_ID = pa_PRODUCT_ID and POLHSH.pl_PARALABH_ID = pa_ID) as OUT_KILOS From PARALABH, PRODUCTS WHERE pa_company_id=1 GROUP BY pa_ID, pa_PRODUCT_ID,pr_ID, pr_NAME HAVING pa_ID=241 and pr_id=pa_PRODUCT_ID
Thanks in advance
Consider avoiding the correlated subquery which runs a SUM separately for each row and use a join of two aggregate queries each of which runs SUM once by grouping fields. Additionally, use explicit joins, the current SQL standard in joining tables/views.
Please adjust column aliases and names to actuals as assumptions were made below.
SELECT t1.*, t2.OUT_KILOS
FROM
(SELECT pa.pa_ID,
pa.pa_PRODUCT_ID,
pr.pr_ID,
pr.pr_NAME,
SUM(pa.pa_KILOS) AS IN_KILOS
FROM PARALABH pa
INNER JOIN PRODUCTS pr
ON pr.pr_id = pa.pa_PRODUCT_ID
WHERE pa.pa_company_id = 1
GROUP BY pa.pa_ID,
pa.pa_PRODUCT_ID,
pr.pr_ID,
pr.pr_NAME
HAVING pa.pa_ID = 241
) AS t1
INNER JOIN
(SELECT POLHSH.pl_PRODUCT_ID,
POLHSH.pl_PARALABH_ID
SUM(pl_KILOS) As OUT_KILOS
FROM POLHSH
GROUP BY POLHSH.pl_PRODUCT_ID,
POLHSH.pl_PARALABH_ID
) AS t2
ON t2.pl_PRODUCT_ID = t1.pa_PRODUCT_ID
AND t2.pl_PARALABH_ID = t1.pa_ID
Related
I am migrating a desktop application from MS Access VBA to Java Springboot application and I need to make a VBA query work in MySQL. The VBA query is insanely big so I present to you a smaller example to demonstrate my situation.
VBA query:
SELECT SELECT tbl_trade.id,
DLookUp("[price]","tbl_so_manifest","[so_id] = "
& tbl_trade.so_id & " AND [product_id] = " & tbl_po_manifest.product_id),
// .... many more attributes....
FROM tbl_so_manifest as sm (((
/// 15 nested INNER JOINS ....
INNER JOIN tbl_trade AS t ON sm.so_id = t.so_id
INNER JOIN tbl_po_manifest AS pm ON sm.product_id = pm.product_id
)));
MySQL conversion:
SELECT SELECT tbl_trade.id,
(SELECT sm.product_id
FROM tbl_so_manifest as sm
INNER JOIN tbl_trade AS t ON sm.so_id = t.so_id
INNER JOIN tbl_po_manifest AS pm ON sm.product_id = pm.product_id),
// ......
FROM tbl_so_manifest as sm (((
/// 15 nested INNER JOINS ....
INNER JOIN tbl_trade AS t ON sm.so_id = t.so_id
INNER JOIN tbl_po_manifest AS pm ON sm.product_id = pm.product_id
)));
but it gives error
Error Code: 1242. Subquery returns more than 1 row 0.844 sec
Suggestions needed on how to handle this.
You can force the sub query to only return one reocrd with a top 1
So this:
DLookUp("[price]","tbl_so_manifest","[so_id] = "
& tbl_trade.so_id & " AND [product_id] = " & tbl_po_manifest.product_id)
Becomes:
(SELECT TOP 1 Price FROM tbl_so_manifest
WHERE so_ID = tbl_trade.so_id AND product_id = sm.product_id
ORDER BY PRICE, ID DESC) AS MyPrice
So you actually don't need a join but simply "pluck" out the value. To restrict the sub query to ONE record, the TOP 1 and the order by "PK" will ensure that only one row is returned. If you leave out the "id", then if two top prices are the same, then you get two rows. By adding the "ID" to the order by, then you will only ever get one row, even if 5 prices are the same.
If other tables in context have same column name, then of course add the table qualifier to the above (but, it wrote it as is due for short hand).
So, as a general rule, you can replace dlookup() with an above type of query.
I not all that familiar with MySQL, so you might not be able to use the table alias as above, but all in all, use a top 1 and order by with "ID (or PK) tossed in after price or whatever in those cases where more then one row could exist. In most cases, a dlookup() should not have multiple values (but dlookup() always only returns one value). So in most "translates" from dlookup() to a sub-query, the top 1 should not be required.
And I can't think that a dlookup() needs a "join". Dlookup() always hits one table, and so should a conversion from dlookup() to a sub query. You want column to pluck, the table, and then the criteria).
If the dlookup() is a FK value, then a left join to the lookup table is easier, will not cause extra rows in the query, but either approach is fine. I find that a sub-query has less chance of "messing" with the main query number of rows returned.
So, a dlookup() to sub-query is a one to one conversion. No join should be required, and as noted in most cases a TOP one + the "id/pk" appended should not be required in most cases.
Consider removing the INNER JOINs of subquery as you do not do it in MS Access' DLookUp (which interestingly you do not use the specified aliases in its criteria logic). Remember SELECT (ironically being first listed clause) is usually the last in SQL's order of operations, so it can read external columns, t.so_id and pm.product_id within the subquery.
...
(SELECT sm.`price`
FROM tbl_so_manifest as sm
WHERE sm.so_id = t.so_id
AND sm.product_id = pm.product_id) AS sub_name
...
More efficiently use subquery as a derived table (or CTE in MySQL 8) to be joined with all other tables. This avoids the row by row calculation and runs look up once to be joined to full resultset.
SELECT
sm.`price`,
...
FROM tbl_so_manifest as sm
/// 15 nested INNER JOINS ....
INNER JOIN tbl_trade AS t ON sm.so_id = t.so_id
INNER JOIN tbl_po_manifest AS pm ON sm.product_id = pm.product_id
INNER JOIN (
SELECT `so_id`, `product_id`, `price`
FROM tbl_so_manifest
) sm
ON sm.so_id = t.so_id
AND sm.product_id = pm.product_id;
Note: MySQL does not require parentheses around multiple JOINs (which hopefully my MS suggestion will one day fix in MS Access SQL).
You can limit the sub query to only return one reocrd using LIMIT 1 key word.
(SELECT Price FROM tbl_so_manifest
WHERE so_ID = tbl_trade.so_id AND product_id = sm.product_id
ORDER BY PRICE, ID DESC LIMIT 1) AS MyPrice
I am working on a query with the following format:
I require all the columns from the Database 'A', while I only require the summed amount (sum(amount)) from the Database 'B'.
SELECT A.*, sum(B.CURTRXAM) as 'Current Transaction Amt'
FROM A
LEFT JOIN C
ON A.Schedule_Number = C.Schedule_Number
LEFT JOIN B
ON A.DOCNUMBR = B.DOCNUMBR
ON A.CUSTNMBR = B.CUSTNMBR
GROUP BY A
ORDER BY A.CUSTNMBR
My question is regarding the grouping statement, database A has about 12 columns and to group by each individually is tedious, is there a cleaner way to do this such as:
GROUP BY A
I am not sure if a simpler way exists as I am new to SQL, I have previously investigated GROUPING_ID statements but thats about it.
Any help on lumped methods of grouping would be helpful
Since the docnumber is the primary key - just use the following SQL:
SELECT A.*, sum(B.CURTRXAM) as 'Current Transaction Amt'
FROM A
LEFT JOIN C
ON A.Schedule_Number = C.Schedule_Number
LEFT JOIN B
ON A.DOCNUMBR = B.DOCNUMBR
ORDER BY RM20401.CUSTNMBR
GROUP BY A.DOCNUMBR
I have a pretty complex SQL query as shown in this fiddle
SELECT payer_payment.payer_id,
Sum(payer_payment.amount) AS total_paid,
Sum(payer_payment.pays * payments_share.single_share) AS fair_share
FROM payers
INNER JOIN (payer_payment
INNER JOIN (SELECT payment_id,
Sum(amount) / Sum(pays) AS single_share
FROM payer_payment
GROUP BY payment_id) AS payments_share
ON payer_payment.payment_id =
payments_share.payment_id)
ON payers.id = payer_payment.payer_id
WHERE payers.user_id = 1
GROUP BY payer_payment.payer_id;
In the fiddle it runs fine on MySQL but when I run it on a SQLite database it either throws an error citing:
(1 no such column: payer_payment.payer_id)
When the column clearly does exist.
or simply returns 0 results, depending on the SQLite implementation (WebSQL vs SQLite.js)
What is the reason for this and is it possible to make my query more database agnostic?
In SQLite, you get an autoincrementing column by using INTEGER PRIMARY KEY.
(And if you rely on the actual values of payers.id in the query, you should give them explicitly.)
You should not try to nest joins when it is not needed:
SELECT payer_payment.payer_id,
Sum(payer_payment.amount) AS total_paid,
Sum(payer_payment.pays * payments_share.single_share) AS fair_share
FROM payers
INNER JOIN payer_payment
ON payers.id = payer_payment.payer_id
INNER JOIN (SELECT payment_id,
Sum(amount) / Sum(pays) AS single_share
FROM payer_payment
GROUP BY payment_id) AS payments_share
ON payer_payment.payment_id = payments_share.payment_id
WHERE payers.user_id = 1
GROUP BY payer_payment.payer_id;
SQLFiddle
I have a mysql query and it works fine when i use where clause, but when i donot use
where clause it gone and never gives the output and finally timeout.
Actually i have used Explain command to check the performance of the query and in both cases the Explain gives the same number of rows used in joining.
I have attached the image of output got with Explain command.
Below is the query.
I couldn't figure whats the problem here.
Any help is highly appreciated.
Thanks.
SELECT
MCI.CLIENT_ID AS CLIENT_ID, MCI.NAME AS CLIENT_NAME, MCI.PRIMARY_CONTACT AS CLIENT_PRIMARY_CONTACT,
MCI.ADDED_BY AS SP_ID, CONCAT(MUD_SP.FIRST_NAME, ' ', MUD_SP.LAST_NAME) AS SP_NAME,
MCI.FK_PROSPECT_ID AS PROSPECT_ID, MCI.DATE_ADDED AS ADDED_ON,
(SELECT GROUP_CONCAT(LT.TAG_TEXT SEPARATOR ', ')
FROM LK_TAG LT
INNER JOIN M_OBJECT_TAG_MAPPING MOTM
ON LT.PK_ID = MOTM.FK_TAG_ID
WHERE MOTM.FK_OBJECT_ID = MCI.FK_PROSPECT_ID
AND MOTM.OBJECT_TYPE = 1
AND MOTM.IS_ACTIVE = 1
) AS TAGS,
IFNULL(SUM(GET_DIGITS(MMR.RCP_AMOUNT)), 0) AS REVENUE_SO_FAR,
IFNULL(SUM(GET_DIGITS(MMR.RCP_RUPEES)), 0) AS REVENUE_INR,
COUNT(DISTINCT PMI_MONTHLY.PROJECT_ID) AS MONTHLY,
COUNT(DISTINCT PMI_FIXED.PROJECT_ID) AS FIXED,
COUNT(DISTINCT PMI_HOURLY.PROJECT_ID) AS HOURLY,
COUNT(DISTINCT PMI_ANNUAL.PROJECT_ID) AS ANNUAL,
COUNT(DISTINCT PMI_CURRENTLY_RUNNING.PROJECT_ID) AS CURRENTLY_RUNNING_PROJECTS,
COUNT(DISTINCT PMI_YET_TO_START.PROJECT_ID) AS YET_TO_START_PROJECTS,
COUNT(DISTINCT PMI_TECH_SALES_CLOSED.PROJECT_ID) AS TECH_SALES_CLOSED_PROJECTS
FROM
M_CLIENT_INFO MCI
INNER JOIN M_USER_DETAILS MUD_SP
ON MCI.ADDED_BY = MUD_SP.PK_ID
LEFT OUTER JOIN M_MONTH_RECEIPT MMR
ON MMR.CLIENT_ID = MCI.CLIENT_ID
LEFT OUTER JOIN M_PROJECT_INFO PMI_FIXED
ON PMI_FIXED.CLIENT_ID = MCI.CLIENT_ID AND PMI_FIXED.PROJECT_TYPE = 1
LEFT OUTER JOIN M_PROJECT_INFO PMI_MONTHLY
ON PMI_MONTHLY.CLIENT_ID = MCI.CLIENT_ID AND PMI_MONTHLY.PROJECT_TYPE = 2
LEFT OUTER JOIN M_PROJECT_INFO PMI_HOURLY
ON PMI_HOURLY.CLIENT_ID = MCI.CLIENT_ID AND PMI_HOURLY.PROJECT_TYPE = 3
LEFT OUTER JOIN M_PROJECT_INFO PMI_ANNUAL
ON PMI_ANNUAL.CLIENT_ID = MCI.CLIENT_ID AND PMI_ANNUAL.PROJECT_TYPE = 4
LEFT OUTER JOIN M_PROJECT_INFO PMI_CURRENTLY_RUNNING
ON PMI_CURRENTLY_RUNNING.CLIENT_ID = MCI.CLIENT_ID AND PMI_CURRENTLY_RUNNING.STATUS = 4
LEFT OUTER JOIN M_PROJECT_INFO PMI_YET_TO_START
ON PMI_YET_TO_START.CLIENT_ID = MCI.CLIENT_ID AND PMI_YET_TO_START.STATUS < 4
LEFT OUTER JOIN M_PROJECT_INFO PMI_TECH_SALES_CLOSED
ON PMI_TECH_SALES_CLOSED.CLIENT_ID = MCI.CLIENT_ID AND PMI_TECH_SALES_CLOSED.STATUS > 4
WHERE YEAR(MCI.DATE_ADDED) = '2012'
GROUP BY MCI.CLIENT_ID ORDER BY CLIENT_NAME ASC
Yes, as many people have said, the key is that when you have the where clause, mysql engine filters the table M_CLIENT_INFO --probably drammatically--.
A similar result as removing the where clause is to to add this where clause:
where 1 = 1
You will see that the performance is degraded also because mysql will try to get all the data.
Remove the where clause and all columns from select and add a count to see how many records you get. If it is reasonable, say up to 10k, then do the following,
put back the select columns related to M_CLIENT_INFO
do not include the nested one "TAGS"
remove all your joins
run your query without where clause and gradually include the joins
this way you'll find out when the timeout is caused.
I would try the following. First, MySQL has a keyword "STRAIGHT_JOIN" which tells the optimizer to do the query in the table order you've specified. Since all you left-joins are child-related (like a lookup table), you don't want MySQL to try and interpret one of those as a primary basis of the query.
SELECT STRAIGHT_JOIN ... rest of query.
Next, your M_PROJECT_INFO table, I dont know how many columns of data are out there, but you appear to be concentrating on just a few columns on your DISTINCT aggregates. I would make sure you have a covering index on these elements to help the query via an index on
( Client_ID, Project_Type, Status, Project_ID )
This way the engine can apply the criteria and get the distinct all out of the index instead of having to go back to the raw data pages for the query.
Third, your M_CLIENT_INFO table. Ensure that has an index on both your criteria, group by AND your Order By, and change your order by from the aliased "CLIENT_NAME" to the actual column of the SQL table so it matches the index
( Date_Added, Client_ID, Name )
I have "name" in ticks as it is also a reserved word and helps clarify the column, not the keyword.
Next, the WHERE clause. Whenever you apply a function to an indexed column name, it doesn't work the greatest, especially on date/time fields... You might want to change your where clause to
WHERE MCI.Date_Added between '2012-01-01' and '2012-12-31 23:59:59'
so the BETWEEN range is showing the entire year and the index can better be utilized.
Finally, if the above do not help, I would consider splitting your query some. The GROUP_CONCACT inline select for the TAGS might be a bit of a killer for you. You might want to have all the distinct elements first for the grouping per client, THEN get those details.... Something like
select
PQ.*,
group_concat(...) tags
from
( the entire primary part of the query ) as PQ
Left join yourGroupConcatTableBasis on key columns
hi I am doing A query to get some product info, but there is something strange going on, the first query returns resultset fast (.1272s) but the second (note that I just added 1 column) takes forever to complete (28-35s), anyone know what is happening?
query 1
SELECT
p.partnumberp,
p.model,
p.descriptionsmall,
p.brandname,
sum(remainderint) stockint
from
inventario_dbo.inventoryindetails ind
left join purchaseorders.product p on (p.partnumberp = ind.partnumberp)
left join inventario_dbo.inventoryin ins on (ins.inventoryinid= ind.inventoryinid)
group by partnumberp, projectid
query 2
SELECT
p.partnumberp,
p.model,
p.descriptionsmall,
p.brandname,
p.descriptiondetail,
sum(remainderint) stockint
from
inventario_dbo.inventoryindetails inda
left join purchaseorders.product p on (p.partnumberp = inda.partnumberp)
left join inventario_dbo.inventoryin ins on (ins.inventoryinid= inda.inventoryinid)
group by partnumberp, projectid
You shouldn't group by some columns and then select other columns unless you use aggregate functions. Only p.partnumberp and sum(remainderint) make sense here. You're doing a huge join and select and then the results for most rows just end up getting discarded.
You can make the query much faster by doing an inner select first and then joining that to the remaining tables to get your final result for the last few columns.
The inner select should look something like this:
select p.partnumberp, projectid, sum(remainderint) stockint
from inventario_dbo.inventoryindetails ind
left join purchaseorders.product p on (p.partnumberp = ind.partnumberp)
left join inventario_dbo.inventoryin ins on (ins.inventoryinid = ind.inventoryinid)
group by partnumberp, projectid
After the join:
select T1.partnumberp, T1.projectid, p2.model, p2.descriptionsmall, p2.brandname, T1.stockint
from
(select p.partnumberp, projectid, sum(remainderint) stockint
from inventario_dbo.inventoryindetails ind
left join purchaseorders.product p on (p.partnumberp = ind.partnumberp)
left join inventario_dbo.inventoryin ins on (ins.inventoryinid = ind.inventoryinid)
group by partnumberp, projectid) T1
left join purchaseorders.product p2 on (p2.partnumberp = T1.partnumberp)
Is descriptiondetail a really large column? Sounds like it could be a lot of text compared to the other fields based on its name, so maybe it just takes a lot more time to read from disk, but if you could post the schema detail for the purchaseorders.product table or maybe the average length of that column that would help.
Otherswise I would try running the query a few times and see you consistently get the same time results. Could just be load on the database server the time you got the slower result.