Duplicate record in mySQL - mysql

I have a mySQL db with duplicate records, as from the attached image.
I am asking for a query to delete all duplicate records based on date + time, for all tables (foreachtables) in db
Thanks

As far I could see, you dont have autoincrement primary key or foreign key.
If you dont have tables with foreign key or relation between, first you can list all your tables. After that, you can create a temporal "mirror" of one table (for eg, autogrill).
Then you can do a:
INSERT INTO TemporalTable
SELECT DISTINCT
or a
INSERT INTO TemporalTable
SELECT Id, Date, Time FROM autogrill GROUP BY Id, Date, Time HAVING COUNT(*) > 1
.
TRUNCATE or DELETE FROM
without where and then put again your data with
INSERT INTO autogrill
SELECT * FROM TemporalTable
BE AWARE if you have primary keys doing this.

How about you create and STORED PROCEDURE for this?
DELIMITER $$
CREATE PROCEDURE `DeleteDup`()
BEGIN
-- Drops the table.
DROP TABLE bad_temp;
-- Creates a temporary table for distincts record.
CREATE TABLE bad_temp(id INT, name VARCHAR(20));
-- Selects distinct record and inserts it on the temp table
INSERT INTO bad_temp(id,name) SELECT DISTINCT id,name FROM bad_table;
-- Delete All Entries from the table which contains duplicate
-- (you can add also condition on this)
DELETE FROM bad_table;
-- Selects all records from temp table and
-- inserts back in the orginal table
INSERT INTO bad_table(id,name) SELECT id,name FROM bad_temp;
-- Drops temporary table.
DROP TABLE bad_temp;
END$$
DELIMITER ;
Please change tablename and column name to your desired schema.
so when you finish creating your STORED PROCEDURE, you can use it like this:
CALL DeleteDup();

You can export your table using this request :
SELECT * FROM autogrill GROUP BY id
Then, empty your table, and import the export you made before. I don't know another easy way to erase duplicate entries using only a single request.

One easy way to do this is to copy all the distinct records into a new table or an export. Then delete all records in the original table and copy them back in.

Export NULL if table have autoincrement an for source use alias name, example :
INSERT INTO product
SELECT NULL,p.product_sku,
p.product_s_desc,
p.product_desc,
p.product_thumb_image,
p.product_full_image,
p.product_tech_data,
p.product_publish,
p.product_weight,
p.product_weight_uom,
p.product_length,
p.product_width,
p.product_height,
p.product_lwh_uom,
p.product_url,
p.product_in_stock,
p.product_available_date,
p.product_special,
p.create_date,
p.modify_date,
p.product_name,
p.attribute
FROM product AS p WHERE p.product_id=xxx;

Related

Sync result of SQL Server view and table MySQL

I have a table with many columns in SQL Server and I have move part of data into MySQL. I made a view or function on the table in SQL Server and these two databases must be synced once a day through job. Because the data of this view may change every day.
View return a table with 3 columns: (char, varchar, varchar) that none of them are unique or primary key.
My solution is:
create a job
execute view on SQL Server
return result of view
create temp table with 3 column in MySQL
move result view from SQL Server to temp table
move records from temp table to new table one by one if not exist before
delete temp table
To transfer without using the temp table, I wanted to use below type of query but could not find the correct query. That's why I used the temp table:
insert into new_table
values (array of records) where record if not exist in new table.
And for the solution I mentioned above, I used the following query:
insert into new_table
select *
from temp_table
where not exist new_table.column = temp_table.column
Do you have a better suggestion that new records can be fetch and added to previous records?
It should look more like this:
insert into new_table
select *
from temp_table
where not exists (
select 1
from new_table
where new_table.column = temp_table.column
)
or maybe this:
insert into new_table
select *
from temp_table
where not exists (
select 1
from new_table
where new_table.column = temp_table.column
and new_table.column2 = temp_table.column2
and new_table.column3 = temp_table.column3
)

Mysql Insert if the Specific Column is not available for the same Employee

I have a controller to save a record
My Table contains BELOW Fields
This is Must (It has to repeat in LOOP).
I want to insert a record single time in a table for a Employee Id, and it should not repeat again. But Same Employee can have multiple Batch Ids and Multiple Course ID's.
If I take Unique as a Employee Id that is not working again to insert the another record to the same employee.
This process should repeat inside the loop and I need to get the Last Inserted ID from the Table and have to assign the number of students in the another table. This everything is working fine if I create a Procedure in Mysql and If I call Procedure. But my Linux server is not executing and throwing MySQL error.
Here is my query and
<code>
$insert_staff_assign = "insert into staff_assign
(`main_course_id`, `main_batch_id`, `section`, `semester_course_id`, `emp_mas_staff_id`, `emp_category`)
VALUES
(:main_course_id, :main_batch_id, :section_id, :semester_course_id, :emp_mas_staff_id, :emp_category)
ON DUPLICATE KEY UPDATE
main_course_id=:main_course_id, main_batch_id=:main_batch_id, section=:section_id, semester_course_id=:semester_course_id, emp_mas_staff_id=:emp_mas_staff_id, emp_category=:emp_category ";
insert into staff_assign
(`main_course_id`, `main_batch_id`, `section`, `semester_course_id`, `emp_mas_staff_id`, `emp_category`)
VALUES
(:main_course_id, :main_batch_id, :section_id, :semester_course_id, :emp_mas_staff_id, :emp_category)
ON DUPLICATE KEY UPDATE
main_course_id=:main_course_id, main_batch_id=:main_batch_id, section=:section_id, semester_course_id=:semester_course_id, emp_mas_staff_id=:emp_mas_staff_id, emp_category=:emp_category
insert into staff_assign
(`main_course_id`, `main_batch_id`, `section`, `semester_course_id`, `emp_mas_staff_id`, `emp_category`)
SELECT * FROM (
SELECT
:main_course_id, :main_batch_id, :section_id, :semester_course_id, :emp_mas_staff_id, :emp_category
) AS tmp WHERE NOT IN (
SELECT emp_mas_staff_id FROM staff_assign WHERE emp_mas_staff_id = $save_emp_mas_staff_id
) LIMIT 1
</code>
Please send me the query to get rid of this problem.
The above are my queries.
I found the answer for the above question.
mysql.proc problem.
In my mysql my mysql.proc was corrupted. That is the reason it doesn't execute the above query.
To fix the above issue you need to update mysql in linux.

update a table when new data is available in another table

I have two table:
Am using this query to populate my marks table with all 'id' in student table:
$module1= "insert into marks (stud_id,moduleID)
select sid, 1
from user_student
where sid not in (
select * from (
select distinct stud_id from marks where moduleID=1
) alias_name
)";
So i need to take all 'sid' from Table student and populate them into table marks my query above does all that. The problem am encountering is that every time i make a change to the data e.g column test1 , new record are inserted again.
If i populate the marks table manually, Can't i have a code that when new data is available in student table, the data is just updated in marks table...
I don't understand some points of your insert query, why insert sid in stud_id if there's a id in student table to relate them?
Maybe solve your problem the creation of a composed unique key to moduleID and stud_id and use a REPLACE in place of INSERT.
But the right way to do it is using a TRIGGER like:
DROP TRIGGER IF EXISTS `student_after_insert`;
CREATE TRIGGER `student_after_insert` AFTER INSERT ON `student`
FOR EACH ROW INSERT INTO marks (stud_id,moduleID)
SELECT NEW.`id`, `modules_table`.`id` FROM `modules_table`;
PS: I suppose you have a table of modules with name modules_table. Change it accordingly.
I haven't understand your question at the best, my tip is:
Make a unique constraint on marks table (up to you is the choice of best constraint rule).
Then use a trigger to make the right insert/update:
Something like that, (hope that syntax is right):
CREATE TRIGGER myTrigger
AFTER INSERT ON students
FOR EACH ROW
BEGIN
// check that insertion doesn't broke the constraint previously define, then
insert into marks (stud_id,moduleID) values (new.sid, 1);
END
Note: NEW is the row that you have as soon inserted/updated on students. Similar you have OLD metarow.

Trouble with MAX(col)+1 INSERT into same MySQL table

I need two id columns in the same table that create unique values upon insert. Since MySQL allows only one column to auto-increment, I need to do something like max(id)+1 for the other column.
Here's the SQL I expected to work:
INSERT INTO invoices (invoiceid)
VALUES ((SELECT MAX(invoiceid)+1 FROM invoices))
The select statement works independently, but within my INSERT, it's not allowed. I get the error : You can't specify target table 'invoices' for update in FROM clause
You want to use INSERT INTO .... SELECT FROM instead of INSERT INTO...VALUES():
INSERT INTO invoices (invoiceid)
SELECT MAX(invoiceid)+1
FROM invoices
My question for you would be why are you not use an AUTO INCREMENT field to generate the invoiceid value? That is what it is for, then you will not have to create this when inserting data.

Copy data from one table into other table in MYSQL

INSERT INTO delete_subscriber (SubID,RandomCode,Email,FirstName,LastName,CreateDate,UpdateDate)
SELECT subscriber.*
FROM subscriber, list_sub
WHERE ListID=? AND list_sub.SubID=subscriber.SubID;
I use a select statement to get all the record and store it into the other database, the problem is , there are more column in the delete_subscriber, that means there has a column e.g. delete date, delete person that is not in subscriber table.
How can I add them in one sql statement ? Or i have to use another statement?
But if i try the latter method, the problem is delete date, delete person are not null, I can not insert the partial record using the 1st statement
INSERT INTO delete_subscriber (SubID,RandomCode,Email,FirstName,LastName,CreateDate,UpdateDate,DeleteDate,DeletePerson)
SELECT s.SubID,s.RandomCode,s.Email,s.FirstName,s.LastName,s.CreateDate,s.UpdateDate,NOW(),?
FROM subscriber s , list_sub
WHERE list_sub.ListID=? AND list_sub.SubID=s.SubID;
I can run it on sql but not php pdo? i found it not doing transaction
Change your insert statement to
INSERT INTO delete_subscriber
(SubID,RandomCode,Email,FirstName,LastName,
CreateDate,UpdateDate,DeleteDate,DeletingPerson)
SELECT s.SubID,s.RandomCode,s.Email,s.FirstName,s.LastName,
s.CreateDate,s.UpdateDate,NOW(),?
FROM subscriber s , list_sub
WHERE list_sub.ListID=? AND list_sub.SubID=s.SubID;
You can Alter delete_subscriber table set delete date, delete person default null before insert.and after populating data from select query modify again them as previous, not null.