Trying to remove duplicate based on three column like query will not find duplicate based on single column it will concatenate three column then remove duplicate rows based on the merge column.
I would appreciate if someone can share an easiest way of achieving this. I know this is not an appropraiote way but tried and its not working
Select concat(col1, col2, col3,) as newCol distinct newcCol from Table2
I know how to remove the table duplicate based on multiple columns using Excel VBA but do not know how to achieve this using mysql
Sub DelDupl()
Range("A1").CurrentRegion.RemoveDuplicates Columns:=Array(1, 2, 3), Header:=xlYes
End Sub
Table name is Table2 in Mysql
enter image description here
Sample Data
CREATE TABLE Table2(
col1 INT,
col2 varchar(10),
col3 INT,
col4 varchar(10),
col5 varchar(10),
col6 varchar(10),
col7 varchar(10));
INSERT INTO Table2 (col1,col2,col3,col4,col5,col6,col7)
VALUES ('1','A','123456','data1','data1','data1','data1'),
('2','B','78910','data2','data2','data2','data2'),
('3','C','45698','data3','data3','data3','data3'),
('1','A','123456','data1','data1','data1','data1'),
('2','B','78910','data2','data2','data2','data2'),
('3','C','45698','data3','data3','data3','data3'),
('4','D','85969','data5','data5','data5','data5');
The problem is there is no way to establish which to keep so I suggest you set up a staging table with a compound key, load it , truncate your existing table and load it from the staging table
How about this:
SELECT *
FROM table2 AS t
GROUP BY t.col1, t.col2, t.col3;
GROUP BY is normally used for aggregating functions(count, sum, max, etc), but will do the job for your purpose.
Edit:
Ok, since you actually need to delete duplicates, thats a bit more complicated, but it's possible. First we need to somehow discriminate duplicate rows, so we will temporary add a primary key. Then execute delete statement, while joining the table on itself to find duplicated rows. And lastly drop the primary key column we added.
add primary key column, we need some discrimination for duplicates
ALTER TABLE `table2` ADD COLUMN id BIGINT NOT NULL AUTO_INCREMENT PRIMARY KEY;
join table on itself to get duplicates and delete them
DELETE t1
FROM `table2` t1
INNER JOIN `table2` t2
ON t1.col1 = t2.col1 AND t1.col2 = t2.col2 AND t1.col3 = t2.col3
WHERE t1.id < t2.id;
drop the primary key column
ALTER TABLE `table2` DROP COLUMN id;
As a note, the join is done on the columns which define duplication(col1, col2, col3) in your case. You can execute all 3 queries at once, just be sure you really need this data gone.
I just create a federated table (cron_task_sync) get the updated data from server. So now I want to update the outdated table (cron_task) in my local mysql database with that table.
Any sql can do this? I found there's a lot of limitation when I use mysql. For instance except cannot be used.
I am sure that that two table are in the same structure. Please help me.
If you have a primary key, like id, you could use not in:
insert into cron_task
(id, col1, col2, col3, ...)
select id
, col1
, col2
, col3
from cron_task_async
where id not in
(
select id
from cron_task
)
I'm going to do an regression analysis through my data chunks. For that I need to find out various values. For each data set I need to get N:count(X) sumX sumY sumX*X etc.
Separately I wrote queries for those operations like
SELECT COUNT(X) FROM table_name
SELECT SUM(X*X) FROM table_name
I need to create another table which a row contain count(X), sumX , sumX*X etc. How can I write that kind of query?
You can add multiple aggregates to the same query and use create table as:
create table yournewtable as
select count(x) cnt, sum(x*x) sumxx, sum(x) sumx
from table_name
SQL Fiddle Demo
This will return you a single row. If you need to break it apart, look into group by.
CREATE TABLE first and then use INSERT INTO
CREATE TABLE yourTableName
(
col1 int,
col2 int,
col3 int
);
INSERT INTO yourTableName (col1, col2, col3)
SELECT
(SELECT COUNT(X) FROM table_name),
(SELECT SUM(X) from table_name),
(SELECT SUM(X*X) from table_name)
I have two tables with same elements, the only differnce is there ids' which are primary key and auto increment.
Table1 | Table2
id1(PK)| id2(PK)
col1 | col1
col2 | col2
col3 | col3
I know some quick ways to do that like,
INSERT INTO table2 SELECT * FROM table1 where id1 = 2
while using such method the content of table2 has id2 = 2 as it copies all the fields directly to table2 from table1, to restric that,
I can also use a method
INSERT INTO table2(col1,col2,col3) SELECT col1,col2,col3 FROM table1 WHERE id1 = 2
such way is good for short tables, but I have lot of columns in my table.
I need a quick way to copy all the columns from table1 to table2 leaving the primary columns which is id2, as it is autoincremented.
Its like I want to copy a specified row from table1 to table2 with different id2(which will be generated as its autoincremented).
Are there any possibilities.
If you do not want to mention column names but want to copy all, then try copy all data into a temp table, then drop pk_id field, and then copy rest of the fields into desired table, lastly drop the temp table.
Refer to one of my answers to a similar queries:
Mysql: Copy row but with new id
We can use temporary table to buffer first from main table and use it to copy to main table again. We can drop the pk field from the temp table and copy all other to the second table.
With reference to the answer by Tim Ruehsen in a referred posting:
CREATE TEMPORARY TABLE tmp_table SELECT * from first_table WHERE ...;
ALTER TABLE tmp_table drop pk_id; # drop autoincrement field
# UPDATE tmp_table SET ...; # just needed to change other unique keys
INSERT INTO second_table SELECT 0, tmp_table.* FROM tmp_table;
DROP TABLE tmp_table;
Copy all rows from table1 except that have same id in table2
INSERT INTO table2(col1, col2, col3)
SELECT t1.col1, t1.col2, t1.col3
FROM table1 t1
WHERE t1.id1 not in (SELECT id2 FROM table2);
I've done some research but all of the examples that I've found seem too complicated given what I would like to do. I have multiple tables of archived data by year (e.g. archive_2013, archive_2012, etc.) and would like to create a new master table (archive_master) consisting of all of the data from all of the tables. The tables have no keys and only 2 columns, one varchar(120) and the other char(20). I'm hoping that this is as simple and straightforward as I think it is.
A simple UNION will do the trick:
SELECT col1, col2
FROM archive_2013
UNION ALL
SELECT col1, col2
FROM archive_2012
Combine it with an INSERT and you are done:
INSERT INTO full_archive
SELECT col1, col2
FROM archive_2013
UNION ALL
SELECT col1, col2
FROM archive_2012
So you want to create one new table?
You could use a simple INSERT INTO with a SELECT
See:
CREATE TABLE with SELECT
http://dev.mysql.com/doc/refman/5.0/en/create-table-select.html
INSERT INTO TABLE with SELECT
http://dev.mysql.com/doc/refman/5.1/en/insert-select.html
Example
You can create a new table:
create table 'xyz' select * from archive_2010;
insert into xyz select * from archive_2011;
INSERT INTO archive_master VALUES (SELECT * FROM archive_2013);