I need to update a column with values from another table ... but it takes forever or aborts. A specific security (isin) can be listed at multiple exchanges (mic) ... so I think I need to have two conditions in the INNER JOIN ... ON (??). Are my attempts below correct? I have about 170,000 records in the table with 40,000 unique isins.
First try:
SQL:
SET SESSION SQL_BIG_SELECTS = 1;
UPDATE securities_live t1
INNER JOIN securities_prev t2
ON t1.isin = t2.isin AND t1.mic = t2.mic
SET t1.prev_close = t2.close;
Second try:
SQL:
SET SESSION SQL_BIG_SELECTS = 1;
UPDATE securities_live t1
INNER JOIN securities_prev t2
ON (t1.isin = t2.isin AND t1.mic = t2.mic)
SET t1.prev_close = t2.close;
Edit regarding indexes for both tables at the moment:
Indexes (securities_live):
Primary|Unique=Yes|Packed=no|Column=id|Cardinality=166776|Collation=A
Indexes (securities_prev):
Primary|Unique=Yes|Packed=no|Column=id|Cardinality=166776|Collation=A
In both tables I have a primary key on column 'id'. So e.g. in table securities_live 'Create a new index' one for column isin and another one for column mic? What about Index name and index type (Primary, Index, Unique, Fulltext)? Size?
For this query:
UPDATE securities_live t1 INNER JOIN
securities_prev t2
ON t1.isin = t2.isin AND t1.mic = t2.mic
SET t1.prev_close = t2.close;
I would suggest an index on securities_prev(isin, mic, close).
However, I suspect that you are updating all or almost all records. If that is the case, it is usually faster to truncate the table and re-populate it with insert. Update is best used for updating a relatively small number of rows.
As the commenters have noted, since you have your indexes set, I suggest you try to do these piecemeal. Do them at 5k a time until it is complete. Try 10k. Obviously you cannot do 170k or 40k. I have had many times where a database had millions of rows, and I had to to them 100k or less at a time. This was because of the limitations of the hardware.
For example,
UPDATE top(1000) securities_live t1
INNER JOIN securities_prev t2
ON t1.isin = t2.isin AND t1.mic = t2.mic
SET t1.prev_close = t2.close;
You may wish to use Order By so you know what records are what and you need keep track of what has been updated.
See here,
how can I Update top 100 records in sql server
Sorry, I just read you were using MySQL,
MySQL - UPDATE query with LIMIT
Try This:
SET SESSION SQL_BIG_SELECTS = 1;
UPDATE securities_live t1
INNER JOIN securities_prev t2
ON (t1.isin = t2.isin AND t1.mic = t2.mic)
SET t1.prev_close = t2.close;
Related
I am not sure how to ask this question - it might be answered somewhere, but it's kind of messy ;)
There are two tables that sort flags (labels) for objects somewhere else.
Table 1: the flags in booka_flags
columns of interest are id(INT) and title(VARCHAR)
Table 2: the flag items in booka_flags_items
columns of interest are id(INT) and flag(VARCHAR)
I want to change booka_flags_items.flag from VARCHAR (which is available from booka_flags.title) to ID (which is available from booka_flags.id)
My "idea" of writing this query (after some research) was the following:
UPDATE
booka_flags_items
SET
booka_flags_items.flag = booka_flags.id
FROM
booka_flags_items
INNER JOIN
authors
ON
booka_flags_items.flag = booka_flags.title
but it does not work. Also not working is this query:
UPDATE
booka_flags_items
SET
booka_flags_items.flag = (
SELECT booka_flags.id
FROM booka_flags
INNER JOIN booka_flags_items
ON booka_flags_items.flag = booka_flags.title
)
What would be the right way to solve a query like that?
I'm guessing you would like to update id column of booka_flags with id column of booka_flags where title matches flags. Your updated query looks incorrect as per mysql
Try:
UPDATE booka_flags_items t1 INNER JOIN booka_flags t2 On t1.flag = t2.title
SET t1.id = t2.id
I'm running a two table inner join query on MySQL server and it is very slow, it takes 3 hours to finish.
UPDATE table_A ta
JOIN table_B tb
ON ta.field1= tb.field1
AND ta.field2 = tb.field2
SET ta.field2 = tb.field2,
ta.field3 = tb.field3
WHERE tb.field5 = 'ABC'
table_A has 650,000 rows
table_B has 100,000 rows
Both ta.field1, ta.field2, tb.field1, tb.field2, tbfield5 are all indexed and shown in the EXPLAIN results.
Any idea how to make it run faster?
For this query:
UPDATE `table_A` ta INNER JOIN
table_B tb
ON ta.`field1`= tb.`field1` AND ta.`field2` = tb.`field2`
SET ta.`field2` = tb.`field2`,
ta.`field3` = tb.`field3`
WHERE tb.`field5` = 'ABC';
There is no need to update field2, given that the new value is the same as the old value (based on the join condition).
One of these index approaches should work best:
table_B(field1, field2, field5)
table_B(field5, field1, field2), table_A(field1, field2)
Which works better depends on the nature of the data. It is very important, though, that you are using composite indexes, not separate indexes on each field.
This is the following code:
UPDATE cm_object
LEFT JOIN (SELECT T1.pandbouwjaar,T.key_id
FROM (SELECT cm_object.key_id,build_year,zipcode,housenumber,housenumber_addition
FROM cm_object
LEFT JOIN cm_key
on cm_object.key_id = cm_key.key_id
) T
LEFT JOIN (SELECT pandbouwjaar,postcode,huisnummer,huisnummertoevoeging
FROM avm_new
GROUP BY postcode,huisnummer,huisnummertoevoeging
) T1
on T.housenumber = T1.huisnummer
AND T.zipcode = T1.postcode
AND T.housenumber_addition = T1.huisnummertoevoeging
) T2
on T2.key_id = cm_object.key_id
SET cm_object.build_year = T2.pandbouwjaar
WHERE cm_object.build_year IS NULL;
I am trying to join two tables cm_object and cm_key. By using the id from cm_key I need to obtain data related to cm_object
Next, I am using the joined table then to link with another table(avm_new) and extract a column from that.
Using that column values, I want to update the build_year column in cm_object
The primary key and indexes have been assigned for all the columns that is being used. Some of the tables have over 8 million records. But the query is taking really long to finish(more than an hour). Why is this happening? Is there a way to optimize it?
I need to update 1 table with data from another where two fields match. I have a query but it's just locking up.
I have an employees_training_courses table
I have a company_training_categories table
I need to get the ID from company_training_categories where both the name and the account_id are the same in both tables.
So far I have this...
update employee_training_courses tc join company_training_categories ctc on ctc.name = tc.name AND ctc.account_id = tc.account_id set tc.company_training_category_id = ctc.id;
i can leave the query running, but it's clearly getting hung up somewhere !
This is your query:
update employee_training_courses tc join
company_training_categories ctc
on ctc.name = tc.name AND ctc.account_id = tc.account_id
set tc.company_training_category_id = ctc.id;
This is a very reasonable query. You probably just need indexes to speed it up. I would recommend:
create index idx_company_training_categories_2 on company_training_categories(name, account_id)
or even:
create index idx_company_training_categories_3 on company_training_categories(name, account_id, id)
I have two tables that need the exact same values for denormalization purposes.
Here's the query.
first table
UPDATE Table_One
SET win = win+1, streak = streak+1, score = score+200
WHERE userid = 1 AND lid = 1 LIMIT 1
second table
UPDATE Table_Two
SET win = win+1, streak = streak+1, score = score+200
WHERE userid = 1 LIMIT 1
As you can see the only difference between both tables is their name and table two doesn't have the field lid
Anyway to combine both updates to just one?
It should be possible with a multi-table update, as described in the documentation.
http://dev.mysql.com/doc/refman/5.5/en/update.html
UPDATE Table_One a INNER JOIN Table_Two b ON (a.userid = b.userid)
SET
a.win = a.win+1, a.streak = a.streak+1, a.score = a.score+200,
b.win = b.win+1, b.streak = b.streak+1, b.score = b.score+200
WHERE a.userid = 1 AND a.lid = 1 AND b.userid = 1
Note: Multi-table doesn't support LIMIT, so this could cause more grief depending on the details.
Stored procedures or transactions may be a nicer solution.
If there is a one to one or one to many relation from Table_One to Table_Two, this would work:
UPDATE Table_One T1, Table_Two T2
SET T1.win = T1.win+1, T1.streak = T1.streak+1, T1.score = T1.score+200,
T2.win = T2.win+1, T2.streak = T2.streak+1, T2.score = T2.score+200
WHERE T1.userid = 1 AND T1.lid = 1 AND T2.userid = T1.userid;
If you can join the tables, then you could create a view of two tables, then update via that view. In your example it looks like userid might be a suitable key.
In creating the view, you'd need to stick to the following guidelines.
They’re two separate queries and so must be treated as such. Sorry to say it, but if you’re updating two tables with identical data, there’s probably a better way to design your database. Remember to keep your programming DRY.
Edit: Should retract that; you can use it for multiple tables, but you can’t use ORDER BY or LIMIT.