I am trying to to split a table into two table and want to add the reference (foreign key) of one table to another
For Example lets say there are is a table called customer table and an Column in it called Group Name has repetitive data which goes against data normalization, so I want to split the table on that column, which would be to replace that Group name Column with GroupID column
NOW form a new table with 2 column GroupID and Group Name and insert distinct Group values into GroupName column, with an auto-incrementing GroupID which will be the Primary key in the group table, and be refrenced in Customer table to act as a foreign key
Now the main question is how to Insert those Autoincremented GroupID of 'GROUP TABLE' into groupID of the customer Table or how do You Insert Foreign Keys into Customer Table According to the Main table
The customerTable With GroupID and customerTable With Customer_Group are two diffrent table
UPDATE ch03.customerd
JOIN ch03.group gr ON customer.Customer_Group = gr.Group_Name
SET customerd.Group_ID = gr.Group_id
WHERE customerd.Customer_id = customer.Customer_id;
update ch03.customerd set Group_ID =
(select gr.Group_id
from ch03.customer as co
join ch03.group as gr on co.Customer_Group = gr.Group_Name)
where customerd.Customer_id = customer.Customer_id;
this is what i have tried and isn't working
If all data except group_id is already transferred then
UPDATE new_customer_table nc
JOIN old_customer_table oc ON nc.customer_id = oc.customer_id
JOIN new_group_table ng ON oc.customer_group = ng.group_name
SET nc.group_id = ng.group_id
If new tables are empty yet then
INSERT INTO new_group_table (group_name)
SELECT DISTINCT customer_group
FROM old_customer_table;
INSERT INTO new_customer_table (customer_id, .. , customer_age, group_id)
SELECT oc.customer_id, .. , oc.customer_age, ng.group_id
FROM old_customer_table oc
JOIN new_group_table ng ON oc.customer_group = ng.group_name;
PS. If customer_group in old table is not defined as ENUM / have no CHECK constraint which checks entered values for validity then I recommend to execute INSERT INTO new_group_table .. firstly and check the values inserted carefully. If some misprintings will be found then you need to edit source data before, then truncate new_group_table and repeat - until all group names are correct.
I have a table with an auto incremented primary key and also a unique key:
CREATE TABLE `product` (
`id` bigint(20) NOT NULL AUTO_INCREMENT,
`canonical_url` varchar(750) CHARACTER SET latin1 NOT NULL,
...
PRIMARY KEY (`id`),
UNIQUE KEY `canonical_url_idx` (`canonical_url`)
Im using the on duplicate key feature to update records if the canonical_url already exists:
"INSERT INTO product(id, canonical_url, name VALUES(?, ? ?) ON DUPLICATE KEY UPDATE name=VALUES(name), id=LAST_INSERT_ID(id)"
KeyHolder productKeyHolder = new GeneratedKeyHolder();
jdbcTemplate.update(conn -> {
PreparedStatement ps = conn.prepareStatement(productSql, new String[] {"id"});
ps.setInt(1, id);
ps.setString(2, canonicalUrl);
ps.setString(3, name);
}, productKeyHolder);
final int productId = productKeyHolder.getKey().intValue();
The problem is that I'm getting this error:
The getKey method should only be used when a single key is returned. The current key entry contains multiple keys: [{GENERATED_KEY=594}, {GENERATED_KEY=595}]
Does anyone know what is causing this?
I just ran into this myself. According to the documentation here:
https://dev.mysql.com/doc/refman/5.0/en/insert-on-duplicate.html
With ON DUPLICATE KEY UPDATE, the affected-rows value per row is 1 if the row is inserted as a new row, and 2 if an existing row is updated.
So when your query executes, if a new record is inserted it's ID is returned. If the record already exists then the existing record is updated. If no update is needed because the values all match then an ID is returned and the number of rows modified is 0. However, if the record is updated, the ID is returned and the number of rows modified is 2. The keyholder is assuming two rows have been modified (even though only one has been) and is incorrectly returning the ID plus the next sequential ID (i.e. the ID plus 1).
To work around this I just checked the count in the getKeys before attempting to call getKey. If there is more than one value in getKeys I won't call getKey.
Assuming that in case of duplicate the temporarily inserted row will have a bigger id, here is a work around :
public static int getGeneratedKeyOnDuplicate(KeyHolder keyHolder)
{
int id = 0;
List<Map<String, Object>> keyList = keyHolder.getKeyList();
if (keyList.size() == 1 && keyHolder.getKey() != null)
id = keyHolder.getKey().intValue();
else if (keyList.size() > 1)
{
id = keyList.stream()
.map((Map<String, Object> key) -> ((Number) key.get("GENERATED_KEY")).intValue())
.min(Comparator.comparing(Integer::valueOf))
.get();
}
return id;
}
But please aware that keyHolder.getKey() is instance of java.math.BigInteger so I am not sure how this will work with id that has a very big value.
Having an issue with this query. (mySQL)
I have a primary key (id) and I want to either insert a new value if the conditions don't exist OR update the existing:
INSERT into records (name, value, p_id, c_id)
VALUES ('Store', 'TX', 1188913, 1133)
ON DUPLICATE KEY UPDATE name = 'TX' WHERE p_id = 6188002 and c_id = 77102
So in this case, I would have a record as such that already exists:
id = 10235192
name = 'Store'
value = 'AL'
p_id = 6188002
c_id = 77102
And I would hope that that record is updated from value = 'AL' to 'TX'
But all I'm seeing is a new record inserted. What am I doing wrong?
You should have specified a UNIQUE constraint on compound column for INSERT...ON DUPLICATE KEY UPDATE to work.
ALTER TABLE records ADD CONSTRAINT tb_uq UNIQUE (p_id, c_id)
and WHERE clause is not needed.
INSERT into records (name, value, p_id, c_id)
VALUES ('Store', 'TX', 1188913, 1133)
ON DUPLICATE KEY UPDATE value = 'TX'
I had the same problem and the answer can be found here.
Conditional duplicate key updates with MySQL
The actual answer is that WHERE clause does not support in MySQL and you need to use IF conditions.
Something like this:
INSERT INTO daily_events
(created_on, last_event_id, last_event_created_at)
VALUES
('2010-01-19', 23, '2010-01-19 10:23:11')
ON DUPLICATE KEY UPDATE
last_event_id = IF(last_event_created_at < VALUES(last_event_created_at), VALUES(last_event_id), last_event_id),
last_event_created_at = IF(last_event_created_at < VALUES(last_event_created_at), VALUES(last_event_created_at), last_event_created_at);
The purpose of this is to copy some rows from one environment to another without overwriting existing rows.
Sample DB:
INSERT INTO `school` (school_id,name) VALUES (15,'Middle');
INSERT INTO `class` (class_id,school_id,name) VALUES (12,15,'Sample');
The idea is school_id and class_id are auto-increments and class has a Foreign Key link back to school. But I want to dump just these rows and insert them into another database that already has a school_id of 15.
It might be something that could look like:
INSERT INTO `school` (name) VALUES ('Middle');
INSERT INTO `class` (school_id,name) VALUES (LAST_INSERT_ID(),'Sample');
But that would just be for this simple example. Imagine if I had 50 classes, 25 students in each, and a few hundred grades for each student/class combo. You could see how the LAST_INSERT_ID() might not work without storing it in a series of variables.
What would be the proper tool to do this kind of operation? Can mysqldump do anything this smart?
You can do this:
Find MAX school_id in the target school table -
SELECT MAX(school_id) INTO #max_school_id FROM school;
Change all school_id values in source tables (school, class) - add MAX school_id from the previous point -
UPDATE school SET school_id = school_id + #max_school_id + 1;
It might be very usefull to add ON UPDATE CASCADE action to the foreign key, it will help to change school_id in the child table automatically, e.g. -
ALTER TABLE class
DROP FOREIGN KEY FK_name;
ALTER TABLE class
ADD CONSTRAINT FK_name FOREIGN KEY (school_id)
REFERENCES school(school_id) ON UPDATE CASCADE;
Make dump and import.
Explanation and example:
Create source tables:
CREATE TABLE school(
school_id INT PRIMARY KEY AUTO_INCREMENT,
name VARCHAR(20)
);
INSERT INTO school (school_id, name) VALUES
(1, 'Middle1'),
(2, 'Middle2'),
(3, 'Middle3'),
(15, 'Middle');
CREATE TABLE class(
class_id INT(11) NOT NULL,
school_id INT(11) DEFAULT NULL,
name VARCHAR(20) DEFAULT NULL,
PRIMARY KEY (class_id),
CONSTRAINT FK_class_school_school_id FOREIGN KEY (school_id)
REFERENCES school (school_id) ON DELETE RESTRICT ON UPDATE CASCADE
)
ENGINE = INNODB;
INSERT INTO class (class_id, school_id, name) VALUES (11, 1, 'Sample1');
INSERT INTO class (class_id, school_id, name) VALUES (12, 15, 'Sample');
Create target tables:
CREATE TABLE school(
school_id INT PRIMARY KEY AUTO_INCREMENT,
name VARCHAR(20)
);
INSERT INTO school (school_id, name) VALUES
(1, 'Top'),
(2, 'Middle'),
(3, 'Bottom'),
(15, 'Top');
CREATE TABLE class(
class_id INT(11) NOT NULL,
school_id INT(11) DEFAULT NULL,
name VARCHAR(20) DEFAULT NULL,
PRIMARY KEY (class_id),
CONSTRAINT FK_class_school_school_id FOREIGN KEY (school_id)
REFERENCES school (school_id) ON DELETE RESTRICT ON UPDATE CASCADE
)
ENGINE = INNODB;
INSERT INTO class (class_id, school_id, name) VALUES (10, 2, 'Sample2');
INSERT INTO class (class_id, school_id, name) VALUES (12, 15, 'Sample');
Update source tables, increment id values:
We should update all unique values, in our case we have to update class_id in the class table and school_id in the school table.
Find max class_id for the TARGET class table
SELECT MAX(class_id) + 1000 FROM class; -- This will return => 1012
Increment all SOURCE class_id values class_id + 1012
UPDATE class SET class_id = class_id + 1012;
Find max school_id for the TARGET school table
SELECT max(school_id) + 1000 FROM school; -- This will return =>1015
Increment all SOURCE school_id values school_id + 1015
UPDATE school SET school_id = school_id + 1015;
That is all. We can dump source tables:
INSERT INTO school VALUES
(1016, 'Middle1'),
(1017, 'Middle2'),
(1018, 'Middle3'),
(1030, 'Middle');
INSERT INTO class VALUES
(1023, 1016, 'Sample1'),
(1024, 1030, 'Sample');
Now we can easily run this script against the target database.
Do you need to do this in SQL? Even the most basic of ETL tools would be better suited. Try pentaho or talend instead.
If you don't need a pure SQL solution you can very easily create a script that reads from the old database and writes to the new. I'm thinking PHP, Python, Ruby, Perl, ...
Here is a simple solution in PHP assuming you are using mysql and moving data between databases (untested, may contain errors):
$dbh1 = new PDO("mysql:host=db1.host;dbname=db1", "user1", "pass1");
$dbh2 = new PDO("mysql:host=db2.host;dbname=db2", "user2", "pass2");
$sth1 = $dbh1->query("
SELECT
school.school_id as school_id,
school.name as school_name,
class.name as class_name
FROM school
JOIN class ON (school.school_id = class.school_id)
");
$sth3 = $dbh2->prepare("INSERT INTO school (name) VALUES (:name)");
$sth4 = $dbh2->prepare("INSERT INTO class (school_id, name) VALUES (:school_id, :name)");
$schools = array();
// get schools and classes
while ($school = $sth1->fetch(PDO::FETCH_ASSOC)) {
$school_id = $school['school_id'];
$school_name = $school['school_name'];
$schools[$school_id]['school_name'] = $school_name;
$schools[$school_id]['classes'][] = array(
'class_name' => $school['class_name']
);
}
// insert schools and classes
foreach ($schools as $school_id => $school) {
// insert school
$sth3->bindParam(':name', $school['school_name'], PDO::PARAM_INT);
$sth3->execute();
$new_school_id = $dbh2->lastInsertId();
// a loop for classes
foreach ($school['classes'] as $class) {
// insert class
$sth4->bindParam(':school_id', $new_school_id, PDO::PARAM_INT);
$sth4->bindParam(':name', $class['class_name'], PDO::PARAM_STR);
$sth4->execute();
}
// a loop for another joined table
/*
foreach ($school['joined'] as $join) {
// insert join
$sth4->bindParam(':school_id', $new_school_id, PDO::PARAM_INT);
$sth4->bindParam(':name', $join['join_name'], PDO::PARAM_STR);
$sth4->execute();
}
*/
}
If you have temporary table privileges you could do:
CREATE TEMPORARY TABLE tmp_school LIKE school;
LOAD DATA LOCAL INFILE 'school.dat' INTO TABLE tmp_school;
CREATE TEMPORARY TABLE tmp_class LIKE class;
LOAD DATA LOCAL INFILE 'class.dat' INTO TABLE tmp_class;
INSERT INTO school (name) SELECT name FROM tmp_school;
INSERT INTO class (school_id,name)
SELECT school.school_id, class.name FROM school school JOIN tmp_school new
USING(name) JOIN tmp_class class ON new.school_id = class.school_id
I think this is right but it needs a bit of checking.
One really simple trick would be to just multiply the id's with -1. Negative id is as good as any id, and I assume that your auto_increment columns start with positive numbers anyways.
Export from one environment:
select -1*school_id, name from school into outfile 'school.out';
select -1*class_id, -1*school_id, name from class into outfile 'class.out';
Import into second:
load data infile 'school.out' into table school;
load data infile 'class.out' into table class;
Obviously this is not a generic solution to your problem, but how often do you need one? :)
Generic solution would be to use write the migration logic yourself, either in ETL tool or just as a standalone script, as others have stated. And make that tool/script to use INFORMATION_SCHEMA.TABLES and INFORMATION_SCHEMA.COLUMNS to dynamically find out the tables and columns that need to be adjusted.
I think the best way is to eliminate the ids from the transfer. assuming that the school names are unique, this seemed to work on deviant's schema, and for simplicity, having the two databases on the same server:
Copy the data into a table on the new database with no ids, just names:
CREATE TEMPORARY TABLE tmp AS SELECT s.name as school_name, c.name as class_name
FROM test.school s JOIN test.class c USING(school_id);
Add the new schools to the school table:
INSERT INTO school (name) SELECT DISTINCT school_name FROM tmp
LEFT JOIN school ON school_name = name WHERE name IS NULL;
Add the new classes (for both existing and new schools) to the class table:
INSERT INTO class (name, school_id) SELECT class_name, school_id
FROM tmp t JOIN school s ON s.name = t.school_name;
What semantics do you want if there are classes in the target databases for schools in the source that don't have that class? This is a union, if you want delete you will have to change it.
if you are using this command in php, then there is a simple function which will give you the last id of your insert query. i.e. mysql_insert_id().
code may be like this :
<?php
$query = mysql_query("INSERT INTO `school` (school_id,name) VALUES (15,'Middle')");
$last_id = mysql_insert_id();
INSERT INTO `class` (class_id,school_id,name) VALUES ('$last_id','Sample');
?>
if you are using in some other language I don't know what has to be done.
I have these two tables just for example:
TAB_TEACHER
- id_teacher // primary key, autoincrement
- name_teacher // a varchar
TAB_STUDENT
- id_student // primary key, autoincrement
- name_student // a varchar
- id_teacher_fk // foreign key reference to a teacher (TAB_TEACHER)
I want to know how to insert in these two cases:
CASE 1 - INSERT a new Student with an pre-existing TEACHER, so I have to get the foreign key with a teacher name
CASE 2 - INSERT a new Student with a new TEACHER (the teacher I'm creating in the same time I'm creating the student)
http://dev.mysql.com/doc/refman/5.0/en/insert-select.html
For case1:
INSERT INTO TAB_STUDENT(name_student, id_teacher_fk)
SELECT 'Joe The Student', id_teacher
FROM TAB_TEACHER
WHERE name_teacher = 'Professor Jack'
LIMIT 1
For case2 you just have to do 2 separate insert statements
Case 1: Insert Row and Query Foreign Key
Here is an alternate syntax I use:
INSERT INTO tab_student
SET name_student = 'Bobby Tables',
id_teacher_fk = (
SELECT id_teacher
FROM tab_teacher
WHERE name_teacher = 'Dr. Smith')
I'm doing this in Excel to import a pivot table to a dimension table and a fact table in SQL so you can import to both department and expenses tables from the following:
Case 2: Insert Row and Then Insert Dependant Row
Luckily, MySQL supports LAST_INSERT_ID() exactly for this purpose.
INSERT INTO tab_teacher
SET name_teacher = 'Dr. Smith';
INSERT INTO tab_student
SET name_student = 'Bobby Tables',
id_teacher_fk = LAST_INSERT_ID()
Case 1
INSERT INTO tab_student (name_student, id_teacher_fk)
VALUES ('dan red',
(SELECT id_teacher FROM tab_teacher WHERE name_teacher ='jason bourne')
it is advisable to store your values in lowercase to make retrieval easier and less error prone
Case 2
mysql docs
INSERT INTO tab_teacher (name_teacher)
VALUES ('tom stills')
INSERT INTO tab_student (name_student, id_teacher_fk)
VALUES ('rich man', LAST_INSERT_ID())