Can anybody please give me a hint on how to optimize this update MySQL query that takes about a minute to process?
UPDATE store s
SET reservation=1
WHERE EXISTS (
SELECT 1
FROM item i
WHERE s.reservation=0
AND s.status!=9
AND s.id=i.store_id
AND i.store_id!=0
)
I need to update (set reservation=1) all rows in "store" table (which is very large) where there is currently reservation=0 but it's id exists in another table "item". Table "item" is also large but not as much as "store".
I'am not an expert on creating efficient queries so forgive me if this is just a completely wrong attitude and the whole thing has a simple solution.
Thanks for any ideas.
It looks like some of the predicates in the correlated subquery could be moved to the outer query. For example, I believe this is equivalent:
UPDATE store s
SET s.reservation = 1
WHERE s.reservation = 0
AND s.status != 9
AND s.id != 0
AND EXISTS ( SELECT 1
FROM item i
WHERE i.store_id = s.id
)
For best performance of that, at a minimum, we'd want an index on store that has reservation as the leading column. Also including the status and id columns would mean those conditions could be checked from the index page, without a lookup of the underlying page in the table.
And for that correlated subquery (dependent query), we'd want an index on item with a store_id as the leading column.
As another option, consider re-writing the correlated subquery as a JOIN operation, for example:
UPDATE store s
JOIN item i
ON i.store_id = s.id
SET s.reservation = 1
WHERE s.reservation = 0
AND s.status != 9
AND s.id != 0
If you're running MySQL 5.5 or earlier, you can't get an EXPLAIN on an UPDATE statement. The closest we can get is rewriting the query as a SELECT, and getting an EXPLAIN on that. MySQL 5.6 does support EXPLAIN on an UPDATE statement.
You can try to use:
UPDATE store s INNER JOIN item i ON s.id=i.store_id SET reservation=1 WHERE i.store_id!=0 AND s.reservation=0 AND s.status != 9;
This case should works faster because you will not go thru all 'item' table each time when you need to check 'store' row.
Related
I need to find a value that exists in LoyaltyTransactionBasketItemStores table but not in DimProductConsolidate table. I need the item code and its corresponding company. This is my query
SELECT
A.ProductReference, A.CompanyCode
FROM
(SELECT ProductReference, CompanyCode FROM dwhdb.LoyaltyTransactionsBasketItemsStores GROUP BY ProductReference) A
LEFT JOIN
(SELECT LoyaltyVariantArticleCode FROM dwhdb.DimProductConsolidate) B ON B.LoyaltyVariantArticleCode = A.ProductReference
WHERE
B.LoyaltyVariantArticleCode IS NULL
It is a pretty straight forward query. But when I run it, it's taking 1 hour and still not finish. Then I use EXPLAIN and this is the result
But when I remove the CompanyCode from my query, its performance is increasing a lot. This is the EXPLAIN result
I want to know why is this happening and is there any way to get ProductReference and its company with a lot more better performance?
Your current query is rife with syntax and structural errors. I would use exists logic here:
SELECT a.ProductReference, a.CompanyCode
FROM dwhdb.LoyaltyTransactionsBasketItemsStores a
WHERE NOT EXISTS (SELECT 1 FROM dwhdb.DimProductConsolidate b
WHERE b.LoyaltyVariantArticleCode = a.ProductReference);
Your current query is doing a GROUP BY in the first subquery, but you never select aggregates, but rather other non aggregate columns. On most other databases, and even on MySQL in strict mode, this syntax is not allowed. Also, there is no need to have 2 subqueries here. Rather, just select from the basket table and then assert that matching records do not exist in the other table.
I have a query with a long list (> 2000 ids) in a WHERE IN clause in mysql (InnoDB):
SELECT id
FROM table
WHERE user_id IN ('list of >2000 ids')
I tried to optimize this by using an INNER JOIN instead of the wherein like this (both ids and the user_id use an index):
SELECT table.id
FROM table
INNER JOIN users ON table.user_id = users.id WHERE users.type = 1
Surprisingly, however, the first query is much faster (by the factor 5 to 6). Why is this the case? Could it be that the second query outperforms the first one, when the number of ids in the where in clause becomes much larger?
This is not Ans to your Question but you may use as alternative to your first query, You can better increase performance by replacing IN Clause with EXISTS since EXISTS performance better than IN ref : Here
SELECT id
FROM table t
WHERE EXISTS (SELECT 1 FROM USERS WHERE t.user_id = users.id)
This is an unfair comparison between the 2 queries.
In the 1st query you provide a list of constants as a search criteria, therefore MySQL has to open and search only table and / or 1 index file.
In the 2nd query you instruct MySQL to obtain the list dynamically from another table and join that list back to the main table. It is also not clear, if indexes were used to create a join or a full table scan was needed.
To have a fair comparison, time the query that you used to obtain the list in the 1st query along with the query itself. Or try
SELECT table.id FROM table WHERE user_id IN (SELECT users.id FROM users WHERE users.type = 1)
The above fetches the list of ids dynamically in a subquery.
I have one query, that is comparatively slow. I tried to rewrite it many times, but I cant find better solution. So I want to ask you, if it is written in wrong way from the beginning or it is ok.
SELECT sql_calc_found_rows
present_id, present_id, present_url, present_name, present_text_short, foto_name, price_id, price_price, price_amount, price_dis
FROM a_present
LEFT JOIN
(SELECT price_id, price_present_id, price_supplier_id, price_dis, price_amount,
(CASE WHEN price_dis <> 0 THEN price_dis ELSE price_amount END) as price_price
FROM a_price
WHERE
price_visibility = 1 AND price_deleted <> 1
GROUP BY price_id ) pri
ON pri.price_present_id = present_id
LEFT JOIN _present_fotos ON foto_id = present_title_foto
LEFT JOIN _cate_pres ON cp_present = present_id
WHERE present_visibility = 1 AND present_deleted <> 1 AND price_price > 0 AND present_out <> 1 AND cp_category IN (30,31,232,32)
GROUP BY present_id
ORDER BY price_price
LIMIT 8
Description: price_dis is price after discount, price_amount is price before discount.. Each product (present) has more prices than one.. Is there faster solution to select final price?
If you will find table structure bad, I will be in trouble:)
Thank you very much!
EDIT:
explain select
OK, so I see a couple of things that could be improved.
First of all, you are JOINing with a table derived from a subquery - with subqueries MySQL does not use indexes (hence the slowdown). Instead of joining with a subquery, try JOINing with a table a_price itself, and put that CASE statement in the original (parent) SELECT. It should allow MySQL to use indexes when JOINing, and it is really important, when your subquery returns many rows.
It should look somewhat like this (including MIN() and GROUP BY, as you need minimum price):
SELECT (...), price_amount, price_dis, MIN((CASE WHEN pri.price_dis <> 0 THEN pri.price_dis ELSE pri.price_amount END)) as price_price
FROM a_present
LEFT JOIN a_price pri
ON pri.price_present_id = present_id AND price_visibility = 1 AND price_deleted <> 1
(...)
GROUP BY present_id
Second of all - as EXPLAIN SELECT suggests - MySQL does not use index on table _cate_pres. You should make it use index to JOIN and to select categories you need (since you put some of them in the IN (..) statement).
Try adding an index on _cate_pres.cp_category, and/or maybe a composite index on this table (using two columns - cp_category and cp_present).
Generally, the result you want to achieve (not always it's possible, but in your case I'm pretty sure it is), is to make the following disappear from EXPLAIN SELECT:
[key] NULL - this means no key is used in this particular set
[Extra] Using temporary - this means a temporary table is created to retrieve results, and it is usually bad for performance
[Extra] Using filesort - this means no index is used for sorting, so sorting process is slow.
Read more about indexes in the Mysql docs, and please give much attention to EXPLAIN output.
How do I optimize the following update because the sub-query is being executed for each row in table a?
update
a
set
col = 1
where
col_foreign_id not in (select col_foreign_id in b)
You could potentially use an outer join where there are no matching records instead of your not in:
update table1 a
left join table2 b on a.col_foreign_id = b.col_foreign_id
set a.col = 1
where b.col_foreign_id is null
This should use a simple select type rather than a dependent subquery.
Your current query (or the one that actually works since the example in the OP doesn't look like it would) is potentially dangerous in that a NULL in b.col_foreign_id would cause nothing to match, and you'd update no rows.
not exists would also be something to look at if you want to replace not in.
I can't tell you that this will make your query any faster, but there is some good info here. You'll have to test in your environment.
Here's a SQL Fiddle illuminating the differences between in, exists, and outer join (check the rows returned, null handling, and execution plans).
Having an issue with an update query taking more than 20 minutes (I kill it after that).
Scenario:
Table one has some 300K records.
Table two contains the same set of records (copied over), but with an extra field that needs to contain the id of the record that matches a number of fields, and has the highest value of another (a score). To clarify, the end result should be table two containing 300K records with each record having the id of another record that has the same set of basic properties, and the highest score within the set of records with those properties.
The below completes in ~5s when I only copy 2K records instead of the full 300k records into table two.
UPDATE vtable2 v1 SET v1.buddy = (
SELECT v2.id FROM vtable1 v2
WHERE
v2.group_id = v1.group_id AND
// 6 more basic comparisons
ORDER BY score DESC LIMIT 1
)
I need to find buddies for the full 300K records. All fields involved in joining and sorting have indexes.
Help much appreciated.
MySQL sub-queries tend to be a little slower. I prefer using joins in such cases. I am not exactly clear on your schema design - but you can try something like this -
UPDATE vtable2 v1
[INNER] JOIN vtable1 v2
ON v2.group_id = v1.group_id
AND //OTHER JOIN CONDITIONS IF ANY
WHERE
//any other conditions
SET
v1.buddy = v2.id
PS - Of-course you need to make sure you have proper indexes on your columns. If you need help with that, you can post the whole query with an explain plan.
you could test with numeric variable
SELECT v2.id FROM vtable1 v2
WHERE
v2.group_id = 1 AND
// 6 more basic comparisons
ORDER BY score DESC LIMIT 1
Anyway I think use Join it's better but I don't have schema DB.
Maybe you have a trouble about index on your sql DB.
You can use an exclusion join to find the row in vtable1 such that no other row in vtable1 with a higher score can be found.
UPDATE vtable2 AS v1
INNER JOIN vtable1 AS v2a ON v1.group_id = v2a.group_id AND (...conditions...)
LEFT OUTER JOIN vtable1 AS v2b ON v1.group_id = v2b.group_id
AND v2a.score < v2b.score AND (...conditions...)
SET v1.buddy = v2.id
WHERE v2b.group_id IS NULL;
You do have to duplicate all the other conditions in the expression for the outer join; you can't put them into the WHERE clause.