What's the best way to dynamically build a column of values?
My PHP app will have the ID fields of a large number of records (<=30k). I need to update these records and I'm wondering if I'm overlooking an easy to do this. What I've considered so far is:
(1) Use a derived table built using UNION ALL.
UPDATE t1
INNER JOIN ( SELECT 1 id UNION ALL
SELECT 2 id UNION ALL
SELECT 3 id UNION ALL
SELECT 4 ) t2 ON t1.id = t2.id
SET t1.status = 'A'
In testing, I've exhausted my memory limit by too many UNION ALLs.
(2) Use a temporary table. The INSERT statement can be built by imploding my array of ID fields.
CREATE TEMPORARY TABLE temp (id INT)
INSERT INTO temp (id) VALUES (1), (2), (3), (4)
UPDATE t1
INNER JOIN temp ON t1.id = temp.id
SET t1.status = 'A'
Is there some other way? Thanks!
That's pretty much it if you are using MySQL.
Some engines support the VALUES clause as a table value constructor(like SQL Server), so you could skip the temporary table step.
select * from (VALUES (1), (2), (3)) q(i);
Similar question here: Table Values() Constructor for Updating Multiple Rows
Looks like you are stuck with a temp table or UNION if you have to UPDATE.
If you can try this (but it will create rows that aren't there)...
INSERT INTO t1 (id, status) VALUES (1, 'A'), (2, 'A'), (3, 'A')
ON DUPLICATE KEY UPDATE status=VALUES(status)
Example SQLFiddle
Related
I have 2 tables (t1 with 581978 rows,t2 with 581196 rows) containing some common fields (f1,f2,f3). The tables normally have the same number of rows, but can get out of sync. When this happens I'd like to identify the rows in t1 that are not in t2. The following python code (just use the 'select stmt' for mysql without python) identified the additional 782 rows in t1 but on my ageing iMac took over 12hrs! Is there a more efficient method?
orphans = ("select t1.f1,t1.f2,t1.f3 from t1 where not exists " \
"(select 1 from t2 where t2.f1=t1.f1 and t2.f2=t1.f2 and " \
"t2.f3=t1.f3)")
cursor.execute(orphans)
for i,(f1,f2,f3) in enumerate(cursor):
print(i+1,"orphans {:<10} {:<10} {<:8}".format(f1,f2,f3))
db-fiddle code (Is there a free db-fiddle, seems I can't save without upgrading to pro?)
CREATE DATABASE if not exists SO_61403216;
use SO_61403216;
drop table if exists t1,t2;
create table t1 (f1 varchar(1),f2 int(2),f3 int(2),f4 int(2));
create table t2 (f1 varchar(1),f2 int(2),f3 int(2),f4 int(2),f5 int(2));
insert into t1 (f1,f2,f3,f4) values ("A",2,3,4);
insert into t1 (f1,f2,f3,f4) values ("B",2,3,4);
insert into t1 (f1,f2,f3,f4) values ("C",2,3,4);
insert into t1 (f1,f2,f3,f4) values ("D",2,3,4);
insert into t2 (f1,f2,f3,f4,f5) values ("A",2,3,44,55);
insert into t2 (f1,f2,f3,f4,f5) values ("B",2,3,99,99);
insert into t2 (f1,f2,f3,f4,f5) values ("C",2,3,99,99);
Query, which exposes the extra row in table t1 using both the "not exist" (preferred output) and "null" methods, which both take 0ms for these small tables, but same query takes hours with 500K+ rows - is there a better/quicker method?
use SO_61403216;
desc t1;
desc t2;
select * from t1;
select * from t2;
select t1.f1,t1.f2,t1.f3 from t1 where not exists (select 1 from t2 where t2.f1=t1.f1 and t2.f2=t1.f2 and t2.f3=t1.f3)
select * from t1 left join t2 on t1.f1=t2.f1 and t1.f2=t2.f2 and t1.f3=t2.f3 where t2.f1 is null;
I am trying to compare multiple column such as part, quantity, and type with another table. I tried with compare part id without problem but when I want to compare part id with quantity it will cause error such as return nothing.
Below coding is for compare part id only.
INSERT INTO `table3`
SELECT *
FROM table1
WHERE NOT EXISTS (SELECT *
FROM table2
WHERE table2.part = table1.part)
How can I compare part and quantity together to find unmatched record.
A handy method in MySQL is in with tuples:
INSERT INTO `table3` ( . . . )
SELECT . . .
FROM table1
WHERE (part, quantity) NOT IN (SELECT part, quantity FROM table2);
More importantly, you should explicitly list all the columns when using an insert. This helps prevent unexpected problems when inserting data.
There are 2 solutions -
1)
INSERT INTO `table3`
SELECT *
FROM table1
WHERE NOT EXISTS (SELECT *
FROM table2
WHERE table2.part = table1.part And table2.quantity=table1.quantity)
2)
INSERT INTO table2 (part, quantity, type)
SELECT part, quantity, type
FROM table1
ON DUPLICATE KEY UPDATE quantity= VALUES(quantity), type= VALUES(type)
Not sure if it's the best approach, but I'd be tempted to try a null-left-join on either a set of values or a concatenation - e.g.
select t1.*
from mytable1 as t1
left join mytable2 as t2 on t1.foo = t2.foo and t1.bar = t2.bar
where t1.id is null;
The only thing to note is that t1.id (and t2.id) are assumed to be the primary key - i.e. can never be null. The end result of the above should be all the rows in t1 that cannot be joined to a row in t2.
I have three tables with contents, now i want to get them and add it into new table but am having this sql error "Column count doesn't match value count at row 1"
here is the sql query.
insert into compare_year(yeara,yearb,yearc,data)
SELECT yeara
FROM table_1
UNION ALL
SELECT yearb, data
FROM table_2
UNION ALL
SELECT yearc
FROM table_3
below is how i created the tables
create table table_1(id int primary key auto_increment,yeara varchar(100));
create table table_2(id int primary key auto_increment,yearb varchar(100),data varchar(100));
create table table_3(id int primary key auto_increment,yearc varchar(100));
my new table is now
create table compare_year(id int primary key auto_increment,yeara varchar(100),yearb varchar(100),yearc varchar(100),data varchar(100))
please can someone help me. thanks
Note:when you union select queries,the number of columns should be equal.
and also you cannot insert mutiple select columns into a single row of another.
My solution will be like:
if three table contain same id,then you can do like this
insert into compare_year(yeara,yearb,yearc,data)
SELECT T1.yeara,T2.yearb,T3.yearc,T2.data
FROM table_1 T1
left Join table_2 T2 on T2.Id = T1.Id
left Join table_3 T3 on T3.Id = T2.Id
It looks like what you want is a JOIN rather than a UNION. When you union two select statements, they must have the same number of fields in the SELECT. For example,
insert into compare_year(yeara)
SELECT yeara
FROM table_1
UNION ALL
SELECT yearb AS yeara
FROM table_2
UNION ALL
SELECT yearc AS yeara
FROM table_3
would be acceptable syntactically. If you want to join the tables,
INSERT INTO compare_year(yeara, yearb, yearc, data)
SELECT table_1.yeara, table_2.yearb, table_3.yearc, table_2.data
FROM table_1, table_2, table_3
but note that this is full cartesian product of the tables. It's likely you want some conditionals as well in a WHERE clause. It's also worth noting that the order of the select cause is what's important for the INSERT, not the field names.
How can I generate the strings 'a' through 'zzz' and then save them to a table using MySQL?
This is a bit difficult to do from "pure" MySQL for two reasons:
No support for recursive queries, forcing you to use temporary table to generate all possible variations.
However, a temporary table can't be referenced multiple times from the same query, so you can't use a Cartesian product (i.e. CROSS JOIN) on it to generate the variations. This forces you to use the "real" table instead of the temporary.
In light of all that, the solution would look like this:
CREATE TABLE TEMP_CHARS (
C CHAR PRIMARY KEY
);
INSERT INTO TEMP_CHARS VALUES('a'), ('b'), ('c'), ('d'), ('e'), ('f'), ('g'), ('h'), ('i'), ('j'), ('k'), ('l'), ('m'), ('n'), ('o'), ('p'), ('q'), ('r'), ('s'), ('t'), ('u'), ('v'), ('w'), ('x'), ('y'), ('z');
INSERT INTO YOUR_TABLE(YOUR_FIELD)
SELECT C
FROM TEMP_CHARS
UNION ALL
SELECT CONCAT(T1.C, T2.C) -- Generate all variations (with repetition) of length 2.
FROM TEMP_CHARS T1 CROSS JOIN TEMP_CHARS T2
UNION ALL
SELECT CONCAT(T1.C, T2.C, T3.C) -- Generate all variations (with repetition) of length 3.
FROM TEMP_CHARS T1 CROSS JOIN TEMP_CHARS T2 CROSS JOIN TEMP_CHARS T3;
DROP TABLE TEMP_CHARS;
Is pretty easy if you aware join = cartesian product,
the below (an alternative version from using a user defined function #Branko Dimitrijevic),
you can put into a text file (join.sql) :-
create table abc ( a char(3) );
create table a ( a char(1) );
insert into a values ('a','b' ..., 'z');
insert into abc select a from a;
insert into abc
select concat(a.a, b.a) from a
join a as b;
insert into abc
select concat(a.a, b.a, c.a) from a
join a as b
join a as c;
And pipe into mysql
mysql -u xxx -h xxx < join.sql
I have a table with some ids + titles. I want to make the title column unique, but it has over 600k records already, some of which are duplicates (sometimes several dozen times over).
How do I remove all duplicates, except one, so I can add a UNIQUE key to the title column after?
This command adds a unique key, and drops all rows that generate errors (due to the unique key). This removes duplicates.
ALTER IGNORE TABLE table ADD UNIQUE KEY idx1(title);
Edit: Note that this command may not work for InnoDB tables for some versions of MySQL. See this post for a workaround. (Thanks to "an anonymous user" for this information.)
Create a new table with just the distinct rows of the original table. There may be other ways but I find this the cleanest.
CREATE TABLE tmp_table AS SELECT DISTINCT [....] FROM main_table
More specifically:
The faster way is to insert distinct rows into a temporary table. Using delete, it took me a few hours to remove duplicates from a table of 8 million rows. Using insert and distinct, it took just 13 minutes.
CREATE TABLE tempTableName LIKE tableName;
CREATE INDEX ix_all_id ON tableName(cellId,attributeId,entityRowId,value);
INSERT INTO tempTableName(cellId,attributeId,entityRowId,value) SELECT DISTINCT cellId,attributeId,entityRowId,value FROM tableName;
DROP TABLE tableName;
INSERT tableName SELECT * FROM tempTableName;
DROP TABLE tempTableName;
Since the MySql ALTER IGNORE TABLE has been deprecated, you need to actually delete the duplicate date before adding an index.
First write a query that finds all the duplicates. Here I'm assuming that email is the field that contains duplicates.
SELECT
s1.email
s1.id,
s1.created
s2.id,
s2.created
FROM
student AS s1
INNER JOIN
student AS s2
WHERE
/* Emails are the same */
s1.email = s2.email AND
/* DON'T select both accounts,
only select the one created later.
The serial id could also be used here */
s2.created > s1.created
;
Next select only the unique duplicate ids:
SELECT
DISTINCT s2.id
FROM
student AS s1
INNER JOIN
student AS s2
WHERE
s1.email = s2.email AND
s2.created > s1.created
;
Once you are sure that only contains the duplicate ids you want to delete, run the delete. You have to add (SELECT * FROM tblname) so that MySql doesn't complain.
DELETE FROM
student
WHERE
id
IN (
SELECT
DISTINCT s2.id
FROM
(SELECT * FROM student) AS s1
INNER JOIN
(SELECT * FROM student) AS s2
WHERE
s1.email = s2.email AND
s2.created > s1.created
);
Then create the unique index:
ALTER TABLE
student
ADD UNIQUE INDEX
idx_student_unique_email(email)
;
Below query can be used to delete all the duplicate except the one row with lowest "id" field value
DELETE t1 FROM table_name t1, table_name t2 WHERE t1.id > t2.id AND t1.name = t2.name
In the similar way, we can keep the row with the highest value in 'id' as follows
DELETE t1 FROM table_name t1, table_name t2 WHERE t1.id < t2.id AND t1.name = t2.name
This shows how to do it in SQL2000. I'm not completely familiar with MySQL syntax but I'm sure there's something comparable
create table #titles (iid int identity (1, 1), title varchar(200))
-- Repeat this step many times to create duplicates
insert into #titles(title) values ('bob')
insert into #titles(title) values ('bob1')
insert into #titles(title) values ('bob2')
insert into #titles(title) values ('bob3')
insert into #titles(title) values ('bob4')
DELETE T FROM
#titles T left join
(
select title, min(iid) as minid from #titles group by title
) D on T.title = D.title and T.iid = D.minid
WHERE D.minid is null
Select * FROM #titles
delete from student where id in (
SELECT distinct(s1.`student_id`) from student as s1 inner join student as s2
where s1.`sex` = s2.`sex` and
s1.`student_id` > s2.`student_id` and
s1.`sex` = 'M'
ORDER BY `s1`.`student_id` ASC
)
The solution posted by Nitin seems to be the most elegant / logical one.
However it has one issue:
ERROR 1093 (HY000): You can't specify target table 'student' for
update in FROM clause
This can however be resolved by using (SELECT * FROM student) instead of student:
DELETE FROM student WHERE id IN (
SELECT distinct(s1.`student_id`) FROM (SELECT * FROM student) AS s1 INNER JOIN (SELECT * FROM student) AS s2
WHERE s1.`sex` = s2.`sex` AND
s1.`student_id` > s2.`student_id` AND
s1.`sex` = 'M'
ORDER BY `s1`.`student_id` ASC
)
Give your +1's to Nitin for coming up with the original solution.
Deleting duplicates on MySQL tables is a common issue, that usually comes with specific needs. In case anyone is interested, here (Remove duplicate rows in MySQL) I explain how to use a temporary table to delete MySQL duplicates in a reliable and fast way (with examples for different use cases).
In this case, something like this should work:
-- create a new temporary table
CREATE TABLE tmp_table1 LIKE table1;
-- add a unique constraint
ALTER TABLE tmp_table1 ADD UNIQUE(id, title);
-- scan over the table to insert entries
INSERT IGNORE INTO tmp_table1 SELECT * FROM table1 ORDER BY sid;
-- rename tables
RENAME TABLE table1 TO backup_table1, tmp_table1 TO table1;