I have been trying to update column from table through raw_sql in ruby on rails as below,
db_connection.execute("update table1 t1 join table2 t2 on t1.s_number = t2.product_id set t1.name = (select name from table3 where mid_size = t2.level)")
Its very slow and taking too much time. Is there any best approach for bulk update in rails through SQL? hope same will be happen if i do from ACTIVERECORD also.
More information table1 having the 1 lac and table2 having 2.5 lacs records
share your thoughts
It is completely dependent on your indexes in both tables.
Try EXPLAIN for your query it will give you insight
It would be great if you share your create table code for both of your tables
Related
I am using MySQL with an INNODB engine on a DigitalOcean machine. The machine has 4GB memory, an 80 GB DISK and 2vCPUs and runs on Ubuntu 16.04.
We have a query that joins three tables that runs very slowly (takes about 5 minutes to return, if it works at all). The size of the tables are 6 million, 20 million and 100 thousand rows, respectively, and there are unique indexes in the tables for each row.
The query looks like this:
SELECT *, table2.column1
FROM table1
INNER JOIN table2 on table1.column1 = table2.column1
INNER JOIN table3 on table1.column2 = table3.column1
WHERE table3.column2 == "{ID}";
We want to embed this query in a data processing / analysis pipeline which dynamically pulls relevant data into memory and then runs further analysis using R. For this purpose, we need to make this query [or an alternative, which does the same thing] run way more efficiently.
Does anyone have any ideas as to how to make this query more efficient, or what the reasons for this slowdown may be? Any help would be greatly appreciated,
Many thanks!
For this query:
select table1.*, table2.column1
from table1 inner join
table2
on table1.column1 = table2.column1 inner join
table3
on table1.column2 = table3.column1
where table3.column2 = "{ID}";
You want indexes on:
table3(column2, column1)
table1(column2, column1)
table2(column1)
So I am very new to MySQL, and I am trying to run a query to update a column if a cell value is present in both tables, and the query is taking forever to run (It's been running for 10 minutes now and no result yet). One of my tables is about 250,000 rows, and the other is about 80,000, so I'm not sure why it is taking so long. The query I am using is:
USE the_db;
UPDATE table1
JOIN table2
ON table2.a = table1.b
SET table1.c = "Y";
I've changed the names of the tables and columns, but the query is exactly the same. I've looked at other answers on here and all of them take a very long time as well. Any help would be appreciated, thanks.
For this query:
UPDATE table1 JOIN
table2
ON table2.a = table1.b
SET table1.c = 'Y';
You want an index on table2(a):
create index idx_table2_a on table2(a);
Also, if there are multiple values of a that match each b, then you could also be generating a lot of intermediate rows, and that would have a big impact on performance.
If that is the case, then phrase the query as:
UPDATE table1
SET table1.c = 'Y'
WHERE EXISTS (SELECT 1 FROM table2 WHERE table2.a = table1.b);
And you need the same index.
The difference between the queries is that this one stops at the first matching row in table2.
When i execute this mysql query like
select * from t1 where colomn1 in (select colomn1 from t2) ,
what really happens?
I want to know if it executes the inner statement for every row?
PS: I have 300,000 rows in t1 and 50,000 rows in t2 and it is taking a hell of a time.
I'm flabbergasted to see that everyone points out to use JOIN as if it is the same thing. IT IS NOT!, not with the information given here. E.g. What if t2.column1 has doubles ?
=> Assuming there are no doubles in t2.column1, then yes, put a UNIQUE INDEX on said column and use a JOIN construction as it is more readable and easier to maintain. If it is going to be faster; that depends on what the query engine makes from it. In MSSQL the query-optimizer (probably) would consider them the same thing; maybe MySQL is 'not so eager' to recognize this... don't know.
=> Assuming there can be doubles in t2.column1, put a (non-unique) INDEX on said column and rewrite the WHERE IN (SELECT ..) into a WHERE EXISTS ( SELECT * FROM t2 WHERE t2.column1 = t1.column1). Again, mostly for readability and ease of maintenance; most likely the query engine will treat them the same...
The things to remember are
Always make sure you have proper indexing (but don't go overboard)
Always realize that what really happens will be an interpretation of your sql-code; not a 'direct translation'. You can write the same functionality in different ways to achieve the same goal. And some of these are indeed more resilient to different scenarios.
If you only have 10 rows, pretty much everything works. If you have 10M rows it could be worth examining the query plan... which most-likely will be different from the one with 10 rows.
A join would be quicker, viz:
select t1.* from t1 INNER JOIN t2 on t1.colomn1=t2.colomn1
Try with INNER JOIN
SELECT t1.*
FROM t1
INNER JOIN t2 ON t1.column1=t2.column1
You should do indexing in column1 and then you can use inner join
for indexing
CREATE INDEX index1 ON t1 (col1);
CREATE INDEX index2 ON t2 (col2);
select t1.* from t1 INNER JOIN t2 on t1.colomn1=t2.colomn1
I've got to update a field in a table (250 000 records) with the value of a field from another table (10 000 000 records) based on the email...
I've tried:
UPDATE table1 t1, table2 t2
SET t1.country = t2.country
WHERE t1.email = t2.email
But I got a "Query is being executed" forever.
What query should I use?
Thanks
This would be a good opportunity to employ a JOIN.
UPDATE table1 as t1
JOIN table2 t2 ON t1.email = t2.email
SET t1.country = t2.country
It will still take a while for your query to process, but it should reduce the time by a significant amount.
I don't see an obvious error in your query (and the database would yield an error in that case). So the problem we're looking at is to speed up the execution of your update. Is one of your two email fields indexed? If not, could you add an index and try again? E.g. ALTER TABLE table2 ADD INDEX(email)
I have mysql queries with a WHERE IN statement.
SELECT * FROM table1 WHERE id IN (1, 2, 15, 17, 150 ....)
How will it perform with hundreds of ids in the IN clause? is it designed to work with many arguments? (my table will have hundreds of thousands of rows and id is the primary field)
is there a better way to do it?
EDIT: I am getting the Ids from the result set of a search server query. So not from the database. I guess a join statement wouldn't work.
I am not sure how WHERE ... IN performes but for me it sounds like a JOIN or maybe a subselect would be the better choice here.
See also: MYSQL OR vs IN performance and http://www.slideshare.net/techdude/how-to-kill-mysql-performance
You should put the IN clause "arguments" into table2 for instance.
Afterwords you make this:
SELECT t1.* FROM table1 t1
INNER JOIN table2 t2 ON t1.Id = t2.Id