How to delete data efficiently without time consume in my sql - mysql

I want to delete data from table_a, using the query below, but it is very slow.
table_a contains 21000 data.
DELETE FROM table_a WHERE id IN (
SELECT
GROUP_CONCAT(N.id)
FROM table_a N
INNER JOIN table_b E ON N.form_id=E.form_id AND N.element_id=E.element_id
WHERE N.option_value=0
AND E.element_type IN('checkbox','radio','select')
)
Is there a more efficient way?

You don't need a subselect. You can directly refer to the table you want to delete from in your statement like this:
DELETE N
FROM table_a N
INNER JOIN table_b E ON N.form_id = E.form_id
AND N.element_id = E.element_id
AND E.element_type IN('checkbox','radio','select')
WHERE N.option_value = 0

Just to offer an alternative to juergen_d's answer, you can also create > insert > rename > drop.
CREATE TABLE temp_source_table LIKE source_table; -- indeces are copied
INSERT INTO temp_source_table SELECT * FROM source_table WHERE <conditions>;
-- you SELECT which data you want to save instead of delete
-- maybe do some checks and balances before continuing with ...
RENAME TABLE source_table TO origin_source_table, temp_source_table TO source_table;
DROP TABLE origin_source_table;
Of course this depends on the usecase of the table in question, but on tables with large amounts of data and/or complex indeces (don't know the tipping point) this could be a faster option.
This would result in:
CREATE TABLE temp_table_a LIKE table_a; -- indeces are copied
INSERT INTO temp_table_a
SELECT N.*
FROM table_a N
LEFT JOIN -- because you select what you want to save
table_b E ON
N.form_id = E.form_id AND N.element_id = E.element_id
WHERE
NOT(N.option_value = 0 AND E.element_type IN('checkbox', 'radio', 'select'));
-- you select which data you want to save instead of delete
-- check/tweak the result of the select before using it as part of the insert
-- maybe do some automatic checks on temp_table_a before continuing with:
RENAME TABLE table_a TO origin_table_a, temp_table_a TO table_a;
DROP TABLE origin_table_a;

Related

i am just tried to create an sql query but not getting

I have two tables. The left side table is Bin and the Right side table is Bout. In_id means an order in which they are batting, Out_id means an order in which they are out from the game. Report1 below shows the answer I want, i.e. those who made a partnership. I am not getting any idea on how to write a query to retrieve the data which is in report 1. That is the answer I actually want. How to write a query for that? Your help would be appreciated!
This is multi step approach. table 1 refers to left side table i.e. batting id table and table 2 refers to right side table i.e out id table in your data.
new.table1, new.table2, new.table3, new.table4 are temporary table. Correct column name as per your data. Hope you will get your result. Comment if find any issue.
Create temporary table new.table1
select OUT.ID, IN.ID, Name, (IN.ID - OUT.ID) AS DIFF from table2;
Create temporary table new.table2
select OUT.ID, IN.ID, Name, (OUT.ID + 1) AS NEW.ID from new.table1 where DIFF <= 0;
Create temporary table new.table3
select OUT.ID, IN.ID, Name, DIFF AS NEW.ID from new.table1 where DIFF > 0;
Create temporary table new.table4
select OUT.ID, IN.ID, Name, NEW.ID from new.table2
UNION ALL
select OUT.ID, IN.ID, Name, NEW.ID from new.table3;
---Final Output
select A.Name, B.Name AS Name2 from new.table4 A, table1 B where A.NEW.ID = B.IN.ID;
select
s1.names,
s2.names
from
IN_TABLE,
OUT_TABLE s1,s2
where
s1.IN_ID = s2.IN_ID;
(where s1 and s2 are alias)
or
select
left_table.name,
right_table.name as Report1
from
left_table,
right_table
where
left_table.IN_ID = right_table.IN_ID
please try and let us know if this solves your problem.
Also, see https://www.youtube.com/watch?v=KTvYHEntvn8 for more knowledge.
You need a join or a subquery.
Lets take a look of a join
Select
leftTable.Name,
rightTable.Name
from
leftTable
join rightTable
on leftTable.IN_ID = rightTable.In_ID
Edited: left to leftTable and right to rightTable

How to increase the performance of table having 50 millions records and growing

We have two tables, table A and table B (contains 50 million records). In table A, we stored a unique value of merchant. From that, we fetched records from table B. How to increase the performance of query and what are the ways to increase performance for it?
Note: The table structure for both tables is simple.
Table A - storage engine (MyISAM)
TABLE B - storage engine (MyISAM)
In table A we have one primary key corresponding to table B's many records.
Queries Used:
Query 1:
records = "select field1, field2... from table A where merchant_id = ''
and field_date between '23-06-2012' and '23-06-2015' order by field 1";
Query 2: execute in loop
foreach (records as records) {
"select field_b1, field_b2, .. from table B where field_b1 =
'records['field1']'"
}
If you haven't done it, add indexes on table_a.merchant_id and table_a.field_date.
Also add indexes on table_2.field_b1.
In addition, you could try to make the main select on table_b, join it with the records from table_a. Something like:
select
B.field_b1, B.field_b2
from table_b AS B
LEFT JOIN table_a AS A on B.field_b1 = A.field_1
WHERE A.merchant_id = '' AND A.field_date between '23-06-2012' and '23-06-2015'
order by field 1;
This way you have only one query for all your records instead of 1 main query + who knows how many additional queries for each record found in the first one.
You should rewrite the logic to be a single query. And, use ISO-standard date formats:
select a.field1, a.field2... , b.field1, b.field2, . .
from a left join
b
on a.field1 = b.field1
where a.merchant_id = '' and
a.field_date between '2012-06-23' and '2015-06-23'
order by a.field1;
For this query, you want indexes on: a(merchant_id, field_date, field1) and b(field1).
Note: if you are only looking for one date, don't use between. Just use =:
select a.field1, a.field2... , b.field1, b.field2, . .
from a left join
b
on a.field1 = b.field1
where a.merchant_id = '' and
a.field_date = '2012-06-23'
order by a.field1;
The query should run faster.

Update a MySQL table to double aggregate of a construct, which depends on the table itself

I need to update a table, but to get the new value it seems that I need to create a temporary table. The reason is that I need to calculate sum of the max. Can I do it?
The pseudocode looks like this:
UPDATE users u SET usersContribution = [CREATE TEMPORARY TABLE IF NOT EXISTS tmpTbl3 AS
(SELECT ROUND(max(zz.zachetTimestamp - d.answerDate)) as answerDateDiff
FROM zachet zz
JOIN discussionansw d ON d.zachetid=zz.zachetId and d.usersid=zz.usersId and
zz.zachetTimestamp > d.answerDate
WHERE zz.whoTalk=u.userid and
NOT EXISTS (SELECT * FROM discussionansw
WHERE zachetid=zz.zachetId and usersid=u.userid')
GROUP BY zz.zachetId)]
SELECT SUM(answerDateDiff) FROM tmpTbl3;"
I used a brackets to show the part, which have to be done, but ignored by UPDATE query...
I have both max and sum and I do not see a way to avoid tmp table. But if you can I we'll be glad to have such a solution.
I put here THE ANSWER, which I get with help of #flaschenpost and this post: SQL Update to the SUM of its joined values
CREATE TEMPORARY TABLE IF NOT EXISTS t0tmpTbl3 AS
(SELECT zz.whoTalk, ROUND(max(zz.zachetTimestamp - d.answerDate)) as answerDateDiff
FROM zachet zz
JOIN discussionansw d ON d.zachetid=zz.zachetId and d.usersid=zz.usersId and
zz.zachetTimestamp > d.answerDate
WHERE
NOT EXISTS (SELECT * FROM discussionansw WHERE zachetid=zz.zachetId and usersid=zz.whoTalk)
GROUP BY zz.zachetId);
UPDATE users u
JOIN (SELECT whoTalk, SUM(answerDateDiff) sumAnswerDateDiff
FROM t0tmpTbl3 GROUP BY whoTalk) t
ON u.usersId=t.whoTalk
SET u.usersContribution=sumAnswerDateDiff;
Could you just break it into two Queries?
drop temporary table if exists tmp_maxsumofsomething;
create temporary table tmp_maxsumofsomething
select max(), sum(), ...
from zachet z inner join discussionansw a on ...
group by...
;
update u inner join tmp_maxsumofsomething t on ... set u.... = t...
Temporary Tables are just visible in the connection where they have been created, so Thread Safety is given.
EDIT: As long as your Queries make any sense, you could try:
DROP TEMPORARY TABLE IF EXISTS tmpTbl3;
CREATE TEMPORARY TABLE tmpTbl3
SELECT zz.whoTalk as userId, ROUND(max(zz.zachetTimestamp - d.answerDate)) as answerDateDiff
FROM zachet zz, discussionansw d
WHERE d.zachetid=zz.zachetId
and d.usersid=zz.usersId and zz.zachetTimestamp > d.answerDate
# What do you mean ? by:
# and (SELECT count(*) FROM discussionansw
# WHERE zachetid=zz.zachetId and usersid=u.userid) = 0
# Think about a reasonable WHERE NOT EXISTS clause!
GROUP BY zz.whoTalk
Then you have your Temp-Table to join to:
update users u
inner join tmpTbl3 tm on u.userId = tm.userId
set u.usersContribution = tm.answerDateDiff
If you are brave enough to write an application needing those queries, you should not be scared to learn a bit more of some concepts of SQL and MySQL. You are here for the exploration of concepts, not to hire Programmers for free.

MySQL Insert from another table with 2 option WHERE statement

I have done my research but can not figure out how to do this. It is super simple to insert from another table but I want to include WHERE statements.
I want to insert value of a single column, column_Q from table A into table B's column_Q WHERE table A's column_W = '100' and column_Q does not already exist in table B.
I tried:
INSERT INTO B (column_Q) select DISTINCT(column_Q)
from A WHERE column_W = 100 AND b.column_Q<>a.column_Q;
Where am I doing wrong?
PS. Both tables already contain values. No field is Null.
INSERT
INTO b (q)
SELECT DISTINCT q
FROM a
WHERE a.w = 100
AND a.q NOT IN
(
SELECT q
FROM b
)
If your b.q has a UNIQUE constraint defined on it, then just use:
INSERT
IGNORE
INTO b (q)
SELECT q
FROM a
WHERE w = 100
You cannot refer to the left side of the "assignment", because there is no current row from B to compare to (that would be the one you are inserting) You need to check if a similar row is already present in B, like in:
INSERT INTO B (column_Q)
SELECT DISTINCT(A.column_Q)
FROM A
WHERE A.column_W = 100
AND NOT EXISTS (
SELECT *
FROM B
WHERE B.column_Q = A.column_Q
);

sql query for deleting rows with NOT IN using 2 columns

I have a table with a composite key composed of 2 columns, say Name and ID. I have some service that gets me the keys (name, id combination) of the rows to keep, the rest i need to delete. If it was with only 1 row , I could use
delete from table_name where name not in (list_of_valid_names)
but how do I make the query so that I can say something like
name not in (valid_names) and id not in(valid_ids)
// this wont work since they separately dont identity a unique record or will it?
Use mysql's special "multiple value" in syntax:
delete from table_name
where (name, id) not in (select name, id from some_table where some_condition);
If your list is a literal list, you can still use this approach:
delete from table_name
where (name, id) not in (select 'john', 1 union select 'sally', 2);
Actually, no I retract my comment about needing special juice or being stuck with (AND OR'ing all your options).
Since you have a list of values of what you want to retain, dump that into a temporary table. Then do a delete against the base table for what does not exist in the temporary table (left outer join). I suck at mysql syntax or I'd cobble together your query. Psuedocode is approximate
DELETE
B
FROM
BASE B
LEFT OUTER JOIN
#RETAIN R
ON R.key1 = B.key1
AND R.key2 = B.key
WHERE
R.key1 IS NULL
The NOT EXISTS version:
DELETE
b
FROM
BaseTable b
WHERE
NOT EXISTS
( SELECT
*
FROM
RetainTable r
WHERE
(r.key1, r.key2) = (b.key1, b.key2)
)