I'm running a two table inner join query on MySQL server and it is very slow, it takes 3 hours to finish.
UPDATE table_A ta
JOIN table_B tb
ON ta.field1= tb.field1
AND ta.field2 = tb.field2
SET ta.field2 = tb.field2,
ta.field3 = tb.field3
WHERE tb.field5 = 'ABC'
table_A has 650,000 rows
table_B has 100,000 rows
Both ta.field1, ta.field2, tb.field1, tb.field2, tbfield5 are all indexed and shown in the EXPLAIN results.
Any idea how to make it run faster?
For this query:
UPDATE `table_A` ta INNER JOIN
table_B tb
ON ta.`field1`= tb.`field1` AND ta.`field2` = tb.`field2`
SET ta.`field2` = tb.`field2`,
ta.`field3` = tb.`field3`
WHERE tb.`field5` = 'ABC';
There is no need to update field2, given that the new value is the same as the old value (based on the join condition).
One of these index approaches should work best:
table_B(field1, field2, field5)
table_B(field5, field1, field2), table_A(field1, field2)
Which works better depends on the nature of the data. It is very important, though, that you are using composite indexes, not separate indexes on each field.
Related
I need to update a column with values from another table ... but it takes forever or aborts. A specific security (isin) can be listed at multiple exchanges (mic) ... so I think I need to have two conditions in the INNER JOIN ... ON (??). Are my attempts below correct? I have about 170,000 records in the table with 40,000 unique isins.
First try:
SQL:
SET SESSION SQL_BIG_SELECTS = 1;
UPDATE securities_live t1
INNER JOIN securities_prev t2
ON t1.isin = t2.isin AND t1.mic = t2.mic
SET t1.prev_close = t2.close;
Second try:
SQL:
SET SESSION SQL_BIG_SELECTS = 1;
UPDATE securities_live t1
INNER JOIN securities_prev t2
ON (t1.isin = t2.isin AND t1.mic = t2.mic)
SET t1.prev_close = t2.close;
Edit regarding indexes for both tables at the moment:
Indexes (securities_live):
Primary|Unique=Yes|Packed=no|Column=id|Cardinality=166776|Collation=A
Indexes (securities_prev):
Primary|Unique=Yes|Packed=no|Column=id|Cardinality=166776|Collation=A
In both tables I have a primary key on column 'id'. So e.g. in table securities_live 'Create a new index' one for column isin and another one for column mic? What about Index name and index type (Primary, Index, Unique, Fulltext)? Size?
For this query:
UPDATE securities_live t1 INNER JOIN
securities_prev t2
ON t1.isin = t2.isin AND t1.mic = t2.mic
SET t1.prev_close = t2.close;
I would suggest an index on securities_prev(isin, mic, close).
However, I suspect that you are updating all or almost all records. If that is the case, it is usually faster to truncate the table and re-populate it with insert. Update is best used for updating a relatively small number of rows.
As the commenters have noted, since you have your indexes set, I suggest you try to do these piecemeal. Do them at 5k a time until it is complete. Try 10k. Obviously you cannot do 170k or 40k. I have had many times where a database had millions of rows, and I had to to them 100k or less at a time. This was because of the limitations of the hardware.
For example,
UPDATE top(1000) securities_live t1
INNER JOIN securities_prev t2
ON t1.isin = t2.isin AND t1.mic = t2.mic
SET t1.prev_close = t2.close;
You may wish to use Order By so you know what records are what and you need keep track of what has been updated.
See here,
how can I Update top 100 records in sql server
Sorry, I just read you were using MySQL,
MySQL - UPDATE query with LIMIT
Try This:
SET SESSION SQL_BIG_SELECTS = 1;
UPDATE securities_live t1
INNER JOIN securities_prev t2
ON (t1.isin = t2.isin AND t1.mic = t2.mic)
SET t1.prev_close = t2.close;
This is the following code:
UPDATE cm_object
LEFT JOIN (SELECT T1.pandbouwjaar,T.key_id
FROM (SELECT cm_object.key_id,build_year,zipcode,housenumber,housenumber_addition
FROM cm_object
LEFT JOIN cm_key
on cm_object.key_id = cm_key.key_id
) T
LEFT JOIN (SELECT pandbouwjaar,postcode,huisnummer,huisnummertoevoeging
FROM avm_new
GROUP BY postcode,huisnummer,huisnummertoevoeging
) T1
on T.housenumber = T1.huisnummer
AND T.zipcode = T1.postcode
AND T.housenumber_addition = T1.huisnummertoevoeging
) T2
on T2.key_id = cm_object.key_id
SET cm_object.build_year = T2.pandbouwjaar
WHERE cm_object.build_year IS NULL;
I am trying to join two tables cm_object and cm_key. By using the id from cm_key I need to obtain data related to cm_object
Next, I am using the joined table then to link with another table(avm_new) and extract a column from that.
Using that column values, I want to update the build_year column in cm_object
The primary key and indexes have been assigned for all the columns that is being used. Some of the tables have over 8 million records. But the query is taking really long to finish(more than an hour). Why is this happening? Is there a way to optimize it?
We have two tables, table A and table B (contains 50 million records). In table A, we stored a unique value of merchant. From that, we fetched records from table B. How to increase the performance of query and what are the ways to increase performance for it?
Note: The table structure for both tables is simple.
Table A - storage engine (MyISAM)
TABLE B - storage engine (MyISAM)
In table A we have one primary key corresponding to table B's many records.
Queries Used:
Query 1:
records = "select field1, field2... from table A where merchant_id = ''
and field_date between '23-06-2012' and '23-06-2015' order by field 1";
Query 2: execute in loop
foreach (records as records) {
"select field_b1, field_b2, .. from table B where field_b1 =
'records['field1']'"
}
If you haven't done it, add indexes on table_a.merchant_id and table_a.field_date.
Also add indexes on table_2.field_b1.
In addition, you could try to make the main select on table_b, join it with the records from table_a. Something like:
select
B.field_b1, B.field_b2
from table_b AS B
LEFT JOIN table_a AS A on B.field_b1 = A.field_1
WHERE A.merchant_id = '' AND A.field_date between '23-06-2012' and '23-06-2015'
order by field 1;
This way you have only one query for all your records instead of 1 main query + who knows how many additional queries for each record found in the first one.
You should rewrite the logic to be a single query. And, use ISO-standard date formats:
select a.field1, a.field2... , b.field1, b.field2, . .
from a left join
b
on a.field1 = b.field1
where a.merchant_id = '' and
a.field_date between '2012-06-23' and '2015-06-23'
order by a.field1;
For this query, you want indexes on: a(merchant_id, field_date, field1) and b(field1).
Note: if you are only looking for one date, don't use between. Just use =:
select a.field1, a.field2... , b.field1, b.field2, . .
from a left join
b
on a.field1 = b.field1
where a.merchant_id = '' and
a.field_date = '2012-06-23'
order by a.field1;
The query should run faster.
I need to update 1 table with data from another where two fields match. I have a query but it's just locking up.
I have an employees_training_courses table
I have a company_training_categories table
I need to get the ID from company_training_categories where both the name and the account_id are the same in both tables.
So far I have this...
update employee_training_courses tc join company_training_categories ctc on ctc.name = tc.name AND ctc.account_id = tc.account_id set tc.company_training_category_id = ctc.id;
i can leave the query running, but it's clearly getting hung up somewhere !
This is your query:
update employee_training_courses tc join
company_training_categories ctc
on ctc.name = tc.name AND ctc.account_id = tc.account_id
set tc.company_training_category_id = ctc.id;
This is a very reasonable query. You probably just need indexes to speed it up. I would recommend:
create index idx_company_training_categories_2 on company_training_categories(name, account_id)
or even:
create index idx_company_training_categories_3 on company_training_categories(name, account_id, id)
Is it possible to achieve next thing without using views, but just one single query? I have two tables:
TableA->TanbleB (1-many) ON TableA.Id = TableB.TableAId
I need to update one field in Table A (TableA.Field1) for records in TableA that satisfy condition on one field in tableA (WHERE TableA.Field2=SomeValue)
.
TableA.Field1 will be updated from TableB with value that is last inserted (last inserted value in related records to TableA).
I will put an example:
UPDATE TableA a SET Field1 = (SELECT TOP 1 b.Feild1 * b.Field2 FROM TableB b WHERE b.TableAId = a.id) WHERE field2 = 1
I know Above example doesn't work, but I have many ways tried using INNER JOIN and failed. I had an idea to use something like this:
UPDATE TableA INNDER JOIN ( SELECT ... FROM TABLE B) ON TABLEA.Id= TableB.TableAId SET ....
But the 2ns query should return 1 record for each DISTINCT TableAId, but only the last inserted.
I hope I am making some sense here.
Thanks in advance.
Here is some SQL that will do what you want
UPDATE T1 INNER JOIN T2 ON T1.ID = T2.T1ID SET T1.F2 = [T2].[F2]*[T2].[F3] WHERE (((T1.F1)="ABC") AND ((T2.ID)=DMax("[ID]","[T2]","[T1ID]=" & [T1].[ID])));
This predicated on T1.ID being the primary key for T1 and T2.T1ID being a index field in T2
One of the flaws in Access is that you can't run an "UPDATE" query based on a "SELECT" query, it will usually give the error:
Operation must use an updateable query
The only way around is as you say to create a view of the "SELECT" query and then inner join this on your table, Access is then working with a static recordset and can handle the "UPDATE" query ok
Alternatively you could write a VBA procedure to step through line by line with the Recordset.
Best of luck : )
UPDATE:
SELECT b.TableAId, b.Feild1 * b.Field2 INTO tblView FROM TableB As b WHERE b.field2 = 1