I'm copying data from one database table to another database table. This essentially is copying the data from our old format to our new format. So, in addition to simply copying columns value-for-value, I also need to do some conversions in the copy statements as well.
For, example, here is what I have to do the copying...
INSERT INTO new_database.table1 (id, product, is_default)
SELECT id, product, is_default FROM old_database.table1
The id and the product are working fine. But, in this example, the old_database stored "is_default" as a VARCHAR(1), either 'Y' or 'N'. The new_database stores "is_default" as a BOOLEAN.
How can I do the conversion between formats within the INSERT SELECT statement I'm already using?
Try like below:
INSERT INTO new_database.table1 (id, product, is_default)
SELECT id, product, IF(is_default='Y',1,0) as isdefault FROM old_database.table1
Related
I have a huge number of data stored in PDF files which I would like to convert into a SQL database. I can extract the tables from the PDF files with some online tools. I also know how to import this into MySQL. BUT:
The list contains users with names, birth dates and some other properties. A user may exist in other PDF files too. So when I'm about to convert the next file into Excel and import it to MySQL, I want to check if that user already exists in my table. And this should be done based on several properties - we may have the same user name, but with different date of birth, that can be a new record. But if all the selected properties match then that specific user would be a duplicate and shouldn't be imported.
I guess this is something I can do with a copy from temporary table but not sure what the selection should be. Let's say user name is stored in column A, date of birth in column B and city in column C. What would be the right script to verify these in the existing table and skip copy if all three match with an existing record?
Thanks!
1- Create a permanent table
Create table UploadData
(
id int not null AUTO_INCREMENT,
name varchar(50),
dob datetime,
city varchar(30)
)
2- Import your data in Excel to your SQL DB. This is how you do it in Sql Server mentioned below, not sure about MySQL but might be something similar. You said you know how to do it already in your question, that's why I am not specifying each step for MySQL
Right-click to your DB, go to Tasks -> Import Data, From: Microsoft Excel, To: Your DB name, Select UploadData table, (check Edit Columns to make sure the columns are matching), finish uploading from Excel to your SQL DB.
3- Check if data exists in your main table, if not, add.
CREATE TEMPORARY TABLE #matchingData (id int, name varchar(50), dob datetime, city (varchar(30))
INSERT INTO #matchingData
select u.id, u.name, u.dob, u.city
from main_table m
inner join UploadData u on u.name = m=name
and u.dob = m.dob
and u.city = m.city
insert into main_table (name, dob, city)
select name, dob, city
from UploadData
where id not in (select id from #matchingData)
4- No need UploadData table anymore. So: DROP TABLE UploadData
Add primary key constraints to Column A, Column B and Column C
It will avoid duplicate rows but can have duplicate values under single column.
Note: There is a limit on maximum number of primary keys in a particular table.
I have a MySql table called Person, and one day I accidentally deleted someone from this table. I have a backup table, called PersonBak so I was going to restore my deletion from the backup. However, in the course of moving forward on my application I renamed all the fields in Person, except for the primary key, PersonID. Now Person and PersonBak have the same data, but only one matching column name.
Is there any way to restore my missing person to Person from PersonBak without doing a lot of work? I have quite a few columns. Of course I could just do the work now, but I can imagine this coming up again.
Is there some way to tell MySql that these are really the same table, with the columns in the same order, just different column names? Or any way at all to do this without writing out specifics of which columns in PersonBak match which ones in Person?
If the column datatypes are the same between the tables, the column count is the same, and they are all in the same order, then MySQL will do all of the work for you:
INSERT INTO t1 SELECT * FROM t2;
The column names are ignored. The server uses ordinal position only, to decide how to line up the from/to columns.
What about this:
insert into Person(id, col11, col12) (select id, col21, col22 from personBak where id=5)
person schema:
columns (id, col11, col12)
personBak schema:
columns (id, col21, col22)
Look at Mysql SELECT INTO and you can specify the field names & create an insert statement
Basically, I have two tables: images and servers. When I want to insert a row into the images table, I need to specify a s_id as one of the fields. Problem is, I only have name, which is another field in the servers table. I need to find what s_id belongs to name, and then use that in my INSERT INTO query on the images table.
Maybe this image will help:
http://i.imgur.com/rYXbW.png
I only know the name field from the servers table, and I need to use it to get the s_id field from the servers table. When I have that, I can use it in my INSERT INTO query, as it's a foreign key.
I found this:
http://www.1keydata.com/sql/sqlinsert.html
But it just confused me even more.
One solution would be to run two queries. One to get the s_id, and one to run the insert query. But I'd like to limit the amount of queries I run if there's a reasonable alternative.
Thanks!
You can use the INSERT ... SELECT form, something like this (with real column names and values of course):
INSERT INTO images (s_id, u_id, name, filename, uploaded)
SELECT s_id, ...
FROM servers
WHERE name = 'the name'
I don't know where you're getting the u_id, name, filename, or uploaded column values for images but you can include them as literal values in the SELECT:
INSERT INTO images (s_id, u_id, name, filename, uploaded)
SELECT s_id, 11, 'pancakes', 'pancakes.jpg', '2011-05-28 11:23:42'
FROM servers
WHERE name = 'the name'
This sort of thing will insert multiple values if servers.name is not unique.
You should be able to do something like this, but you'll need to fill in the items in <> with the values you want to insert.
INSERT INTO images (s_id, u_id, name, filename, uploaded)
(SELECT s_id, <u_id>, <name>, <filename>, <uploaded>
FROM imgstore.servers
WHERE name = #server_name)
This is the syntax for SQL Server, but I think it will work with MySQL as well.
Here's an article on INSERT ... SELECT Syntax
Please see my comment above regarding a potential data integrity issue. I am assuming that the name field in your server table has a unique constraint placed on it.
There are a couple of ways that you can approach this INSERT, and I'm sure that some are better than others. I make no claim that my way is the best way, but it should work. I don't know how you're writing this query, so I'm going to use #FieldValue to represent the variable input. My approach is to use a subquery in your insert statement to get the data that you require.
INSERT INTO images (field1, field2... s_id) VALUES ('#field1val', '#field2val'... (SELECT s_id FROM servers WHERE name='#nameval'));
(MySQL) I am trying to migrate a "subscription" table into 3 new tables: "product", "subscription", "actual" where "actual" is the name of the actual product, say, newsletter. The subscription has a FK reference to product and the actual has FK reference into subscription.
I know the INSERT INTO SELECT statement can copy data probably from many table to one; is there a statement to do the opposite, one to many tables for my case?
I'm not aware of an SQL statement that will do what you want. Just do several INSERT INTO SELECTs one after the other. It may be faster to do them one at a time anyway.
I think you can use three seperate insert into select statements. First you convert the product table, then the subscription where you can use an embedded select to find the id in the product table:
insert into subscription (some_column, FK_id,...)
select something, (select id from product where <your where clause>),...
and finally convert the actual table using an embedded select to get the id from the subscription table.
I'm using MySQL 4.1. Some tables have duplicates entries that go against the constraints.
When I try to group rows, MySQL doesn't recognise the rows as being similar.
Example:
Table A has a column "Name" with the Unique proprety.
The table contains one row with the name 'Hach?' and one row with the same name but a square at the end instead of the '?' (which I can't reproduce in this textfield)
A "Group by" on these 2 rows return 2 separate rows
This cause several problems including the fact that I can't export and reimport the database. On reimporting an error mentions that a Insert has failed because it violates a constraint.
In theory I could try to import, wait for the first error, fix the import script and the original DB, and repeat. In pratice, that would take forever.
Is there a way to list all the anomalies or force the database to recheck constraints (and list all the values/rows that go against them) ?
I can supply the .MYD file if it can be helpful.
To list all the anomalies:
SELECT name, count(*) FROM TableA GROUP BY name HAVING count(*) > 1;
There are a few ways to tackle deleting the dups and your path will depend heavily on the number of dups you have.
See this SO question for ways of removing those from your table.
Here is the solution I provided there:
-- Setup for example
create table people (fname varchar(10), lname varchar(10));
insert into people values ('Bob', 'Newhart');
insert into people values ('Bob', 'Newhart');
insert into people values ('Bill', 'Cosby');
insert into people values ('Jim', 'Gaffigan');
insert into people values ('Jim', 'Gaffigan');
insert into people values ('Adam', 'Sandler');
-- Show table with duplicates
select * from people;
-- Create table with one version of each duplicate record
create table dups as
select distinct fname, lname, count(*)
from people group by fname, lname
having count(*) > 1;
-- Delete all matching duplicate records
delete people from people inner join dups
on people.fname = dups.fname AND
people.lname = dups.lname;
-- Insert single record of each dup back into table
insert into people select fname, lname from dups;
-- Show Fixed table
select * from people;
Create a new table, select all rows and group by the unique key (in the example column name) and insert in the new table.
To find out what is that character, do the following query:
SELECT HEX(Name) FROM TableName WHERE Name LIKE 'Hach%'
You will se the ascii code of that 'square'.
If that character is 'x', you could update like this:(but if that column is Unique you will have some errors)
UPDATE TableName SET Name=TRIM(TRAILING 'x' FROM Name);
I'll assume this is a MySQL 4.1 random bug. Somes values are just changing on their own for no particular reason even if they violates some MySQL constraints. MySQL is simply ignoring those violations.
To solve my problem, I will write a prog that tries to resinsert every line of data in the same table (to be precise : another table with the same caracteristics) and log every instance of failures.
I will leave the incident open for a while in case someone gets the same problem and someone else finds a more practical solution.