This is the following code:
UPDATE cm_object
LEFT JOIN (SELECT T1.pandbouwjaar,T.key_id
FROM (SELECT cm_object.key_id,build_year,zipcode,housenumber,housenumber_addition
FROM cm_object
LEFT JOIN cm_key
on cm_object.key_id = cm_key.key_id
) T
LEFT JOIN (SELECT pandbouwjaar,postcode,huisnummer,huisnummertoevoeging
FROM avm_new
GROUP BY postcode,huisnummer,huisnummertoevoeging
) T1
on T.housenumber = T1.huisnummer
AND T.zipcode = T1.postcode
AND T.housenumber_addition = T1.huisnummertoevoeging
) T2
on T2.key_id = cm_object.key_id
SET cm_object.build_year = T2.pandbouwjaar
WHERE cm_object.build_year IS NULL;
I am trying to join two tables cm_object and cm_key. By using the id from cm_key I need to obtain data related to cm_object
Next, I am using the joined table then to link with another table(avm_new) and extract a column from that.
Using that column values, I want to update the build_year column in cm_object
The primary key and indexes have been assigned for all the columns that is being used. Some of the tables have over 8 million records. But the query is taking really long to finish(more than an hour). Why is this happening? Is there a way to optimize it?
Related
I am not sure how to ask this question - it might be answered somewhere, but it's kind of messy ;)
There are two tables that sort flags (labels) for objects somewhere else.
Table 1: the flags in booka_flags
columns of interest are id(INT) and title(VARCHAR)
Table 2: the flag items in booka_flags_items
columns of interest are id(INT) and flag(VARCHAR)
I want to change booka_flags_items.flag from VARCHAR (which is available from booka_flags.title) to ID (which is available from booka_flags.id)
My "idea" of writing this query (after some research) was the following:
UPDATE
booka_flags_items
SET
booka_flags_items.flag = booka_flags.id
FROM
booka_flags_items
INNER JOIN
authors
ON
booka_flags_items.flag = booka_flags.title
but it does not work. Also not working is this query:
UPDATE
booka_flags_items
SET
booka_flags_items.flag = (
SELECT booka_flags.id
FROM booka_flags
INNER JOIN booka_flags_items
ON booka_flags_items.flag = booka_flags.title
)
What would be the right way to solve a query like that?
I'm guessing you would like to update id column of booka_flags with id column of booka_flags where title matches flags. Your updated query looks incorrect as per mysql
Try:
UPDATE booka_flags_items t1 INNER JOIN booka_flags t2 On t1.flag = t2.title
SET t1.id = t2.id
This query (along with a few others I think have a related issue) did not take 30 seconds when MySQL was local on the same EC2 instance as the rest of the website. More like milliseconds.
Does anything look off?
SELECT *, chv_images.image_id FROM chv_images
LEFT JOIN chv_storages ON chv_images.image_storage_id =
chv_storages.storage_id
LEFT JOIN chv_users ON chv_images.image_user_id = chv_users.user_id
LEFT JOIN chv_albums ON chv_images.image_album_id = chv_albums.album_id
LEFT JOIN chv_categories ON chv_images.image_category_id =
chv_categories.category_id
LEFT JOIN chv_meta ON chv_images.image_id = chv_meta.image_id
LEFT JOIN chv_likes ON chv_likes.like_content_type = "image" AND
chv_likes.like_content_id = chv_images.image_id AND chv_likes.like_user_id = 1
LEFT JOIN chv_follows ON chv_follows.follow_followed_user_id =
chv_images.image_user_id
LEFT JOIN chv_follows_projects ON
chv_follows_projects.follows_project_project_id =
chv_images.image_project_id LEFT JOIN chv_projects ON
chv_projects.project_id = follows_project_project_id WHERE
chv_follows.follow_user_id='1' OR (follows_project_user_id = 1 AND
chv_projects.project_privacy = "public" AND
chv_projects.project_is_public_upload = 1) GROUP BY chv_images.image_id
ORDER BY chv_images.image_id DESC
LIMIT 0,15
And this is what EXPLAIN shows:
Thank you
Update: This query has the same issue. It does not have a GROUP BY.
SELECT *, chv_images.image_id FROM chv_images
LEFT JOIN chv_storages ON chv_images.image_storage_id =
chv_storages.storage_id
LEFT JOIN chv_users ON chv_images.image_user_id = chv_users.user_id
LEFT JOIN chv_albums ON chv_images.image_album_id = chv_albums.album_id
LEFT JOIN chv_categories ON chv_images.image_category_id =
chv_categories.category_id
LEFT JOIN chv_meta ON chv_images.image_id = chv_meta.image_id
LEFT JOIN chv_likes ON chv_likes.like_content_type = "image" AND
chv_likes.like_content_id = chv_images.image_id AND chv_likes.like_user_id = 1
ORDER BY chv_images.image_id DESC
LIMIT 0,15
That EXPLAIN shows several table-scans (type: ALL), so it's not surprising that it takes over 30 seconds.
Here's your EXPLAIN:
Notice the column rows shows an estimated 14420 rows read from the first table chv_images. It's doing a table-scan of all the rows.
In general, when you do a series of JOINs, you can multiple together all the values in the rows column of the EXPLAIN, and the final result is how many row-reads MySQL has to do. In this case it's 14420 * 2 * 1 * 1 * 2 * 1 * 916, or 52,834,880 row-reads. That should put into perspective the high cost of doing several table-scans in the same query.
You might help avoid those table-scans by creating some indexes on these tables:
ALTER TABLE chv_storages
ADD INDEX (storage_id);
ALTER TABLE chv_categories
ADD INDEX (category_id);
ALTER TABLE chv_likes
ADD INDEX (like_content_id, like_content_type, like_user_id);
Try creating those indexes and then run the EXPLAIN again.
The other tables are already doing lookups by primary key (type: eq_ref) or by secondary key (type: ref) so those are already optimized.
Your EXPLAIN shows your query uses a temporary table and filesort. You should reconsider whether you need the GROUP BY, because that's probably causing the extra work.
Another tip is to avoid using SELECT * because it might be forcing the query to read many extra columns that you don't need. Instead, explicitly name only the columns you need.
Is there any indexes in chv_images?
I propose:
CREATE INDEX idx_image_id ON chv_images (image_id);
(Bill's ideas are good. I'll take the discussion a different way...)
Explode-Implode -- If the LEFT JOINs match no more than 1 row, change, for example,
SELECT
...
LEFT JOIN chv_meta ON chv_images.image_id = chv_meta.image_id
into
SELECT ...,
( SELECT foo FROM chv_meta WHERE image_id = chv_images.image_id ) AS foo, ...
If that can be done for all the JOINs, you can get rid of GROUP BY. This will avoid the costly "explode-implode" where JOINs lead to more rows, then GROUP BY gets rid of the dups. (I suspect you can't move all the joins in.)
OR -> UNION -- OR is hard to optimize. Your query looks like a good candidate for turning into UNION, then making more indexes that will become useful.
WHERE chv_follows.follow_user_id='1'
OR (follows_project_user_id = 1
AND chv_projects.project_privacy = "public"
AND chv_projects.project_is_public_upload = 1
)
Assuming that follows_project_user_id is in `chv_images,
( SELECT ...
WHERE chv_follows.follow_user_id='1' )
UNION DISTINCT -- or ALL, if you are sure there won't be dups
( SELECT ...
WHERE follows_project_user_id = 1
AND chv_projects.project_privacy = "public"
AND chv_projects.project_is_public_upload = 1 )
Indexes needed:
chv_follows: (follow_user_id)
chv_projects: (project_privacy, project_is_public_upload) -- either order
But this has not yet handled the ORDER BY and LIMIT. The general pattern for such:
( SELECT ... ORDER BY ... LIMIT 15 )
UNION
( SELECT ... ORDER BY ... LIMIT 15 )
ORDER BY ... LIMIT 15
Yes, the ORDER BY and LIMIT are repeated.
That works for page 1. If you want the next 15 rows, see http://mysql.rjweb.org/doc.php/pagination#pagination_and_union
After building those two sub-selects, look at them; I think you will be able to optimize each one, and may need new indexes because the Optimizer will start with a different 'first' table.
I'm looking for a simple way to do an update on a table only if there is no other columns present in that same table with the same value I'm trying to update, ideally in a single query. So far I'm getting an error You specify target table 't1' for update in FROM clause. Here is what I tried in a few variations so far (still unable to get working):
UPDATE emailQueue AS t1
SET
t1.lockedOn = 1470053240
WHERE
(SELECT
COUNT(*)
FROM
emailQueue AS t2
WHERE
t2.lockedOn = 1470053240) = 0
AND t1.lockedOn IS NULL
In MySQL, you need to use a join. In this case, a left join is in order:
UPDATE emailQueue eq LEFT JOIN
emailQueue eq2
ON eq2.lockedOn = 1470053240
SET eq.lockedOn = 1470053240
WHERE eq.lockedOn IS NULL AND
eq2.lockedOn IS NULL;
I need to update 1 table with data from another where two fields match. I have a query but it's just locking up.
I have an employees_training_courses table
I have a company_training_categories table
I need to get the ID from company_training_categories where both the name and the account_id are the same in both tables.
So far I have this...
update employee_training_courses tc join company_training_categories ctc on ctc.name = tc.name AND ctc.account_id = tc.account_id set tc.company_training_category_id = ctc.id;
i can leave the query running, but it's clearly getting hung up somewhere !
This is your query:
update employee_training_courses tc join
company_training_categories ctc
on ctc.name = tc.name AND ctc.account_id = tc.account_id
set tc.company_training_category_id = ctc.id;
This is a very reasonable query. You probably just need indexes to speed it up. I would recommend:
create index idx_company_training_categories_2 on company_training_categories(name, account_id)
or even:
create index idx_company_training_categories_3 on company_training_categories(name, account_id, id)
Modified some stuff from my pic so you guys can understand it
I have this database. I am trying to update a value from a table based on another value from an another table.
I want to update the SUM from salary like this :
( sum = presence * 5 )
This is what I've been trying to use ( unsuccessful )
update table salary
set suma.salary = users.presence * 5
FROM salary INNER JOIN users1 INNER JOIN presence on id_salary = id_presence
I am not sure what to do, I'd appreciate some help, Thanks
In MySQL to UPDATE tables with a join you use this syntax:
UPDATE table1, table2
SET table1.column = some expression
WHERE table1.column = table2.column
That said, even with the updated picture, in your SQL you are mentioning columns that I cannot understand in which table are to be found. You also have an inner join between salariu and users1, with no join condition. Could you please clean up the question and make everything clear?
Assuming you are making the updates to the db structure you were talking about, then you can start working on this one maybe:
UPDATE salary, presence
SET salary.sum = SUM(presence.hours) * 5
WHERE presence.id = salary.id
AND <some filter on the month that depends on salary.date>
Another way, but I'm not sure it is supported in all RDBMS, would be something like this:
UPDATE salary
SET sum = (
SELECT SUM(presence.hours) * 5
FROM user, presence
WHERE presence.id = salary.id
AND <some filter on the month that depends on salary.date>
)