My lecturer has given me a document with raw data in and I have to create tables and implement it into MySQL.
Background: The raw data has multiple transactions from companies. Some transactions are from the same company so the same COMPANYID comes up often.
The lecturer told us to insert the raw data into a table titled RAW to store everything. Then we have to insert data into our smaller tables from that RAW table.
ISSUE:
My issue is when I’m trying to create my table COMPANY, I obviously want to include the COMPANYID.
But when I use the code
Insert into COMPANY
Select distinct COMPANYID, COMPANYNAME, NumberofDivisions
From RAW;
I get the duplicate error because obviously in the RAW table, the same COMPANYID comes up multiple times for each transaction!
How can I only have the COMPANYID once in the COMPANY table?
Try insert ignore instead of insert which would continue inserting records till the end of file even though duplicate record error.
Insert ignore into COMPANY
Select distinct COMPANYID, COMPANYNAME, NumberofDivisions
From RAW;
Related
A client needs to migrate a large volume of data and I feel this question could be generic enough for SO.
Legacy system
Student profiles contain fields like names, emails etc, as well as university name. The university name is represented as a string and as such is repeated which is wasteful and slow.
Our new form
A more efficient solution is to have a table called university that only stores the university name once with a foreign key (university_id) and the HTML dropdown just POSTs the university_id to the server. This makes things much faster for doing GROUP BY queries, for example. New form data going into the database works fine.
The problem
How can we write a query that will INSERT all the other columns (first_name, last_name, email, ...) but then rather than inserting the university string, find out its university_id from the university table and INSERT the corresponding int instead of the original string? (scenario: data is in a CSV file that we will manipulate into INSERT INTO syntax)
Many thanks.
Use INSERT INTO ... SELECT with a LEFT JOIN. Left is chosen so that student record won't get discarded if it has a null value for university_name.
INSERT INTO students_new(first_name, last_name, email, university_id)
SELECT s.first_name, s.last_name, s.email, u.university_id
FROM students_old s
LEFT JOIN university u ON s.university_name = u.university_name
Table and column names are to be replaced for real ones. Above assumes that your new table for students holding foreign key to university is students_new while the old one (from before normalisation) is students_old.
I have a table called: Orders (Id, FK_Customer_Id, Comments, PaymentMethod, DeliverTime), which is used to store order information of a online retailer website, and also a table: Order_Items (Id, FK_Order_Id, FK_Product_Id, Quantity), to store the details of an order.
once the user submit an order, I need to save the information to the two table, but I don't know how to solve the concurrency problem.
Because, I should first insert into Orders table, then get the Order_Id and insert this Id to Order_Items table. I try to use "SELECT LAST_INSERT_ID() AS OrderId". But what if I insert into order and before I can get the Id, someone else insert another Order? Then the above statement will get the Id of the new inserted order, which is not my order anymore.
Please help me and let me know how to do it, or if I design the order/orderItem in a wrong way.
P.S. I m using angular 2 as front-end and node.js to create APIs and MySQL as DB.
Thank you in advance.
I would separate the operations.
This will prevent any unexpected "race" affects with inserting records.
So, first, insert the order.
Then, second, confirm that the insert query was successful by asking for the "insert id" with something like mysql_insert_id() or whatever the node.js equivalent is.
Then, third, once you have that id, you can safely insert the details of the order.
Basically, I have two tables: images and servers. When I want to insert a row into the images table, I need to specify a s_id as one of the fields. Problem is, I only have name, which is another field in the servers table. I need to find what s_id belongs to name, and then use that in my INSERT INTO query on the images table.
Maybe this image will help:
http://i.imgur.com/rYXbW.png
I only know the name field from the servers table, and I need to use it to get the s_id field from the servers table. When I have that, I can use it in my INSERT INTO query, as it's a foreign key.
I found this:
http://www.1keydata.com/sql/sqlinsert.html
But it just confused me even more.
One solution would be to run two queries. One to get the s_id, and one to run the insert query. But I'd like to limit the amount of queries I run if there's a reasonable alternative.
Thanks!
You can use the INSERT ... SELECT form, something like this (with real column names and values of course):
INSERT INTO images (s_id, u_id, name, filename, uploaded)
SELECT s_id, ...
FROM servers
WHERE name = 'the name'
I don't know where you're getting the u_id, name, filename, or uploaded column values for images but you can include them as literal values in the SELECT:
INSERT INTO images (s_id, u_id, name, filename, uploaded)
SELECT s_id, 11, 'pancakes', 'pancakes.jpg', '2011-05-28 11:23:42'
FROM servers
WHERE name = 'the name'
This sort of thing will insert multiple values if servers.name is not unique.
You should be able to do something like this, but you'll need to fill in the items in <> with the values you want to insert.
INSERT INTO images (s_id, u_id, name, filename, uploaded)
(SELECT s_id, <u_id>, <name>, <filename>, <uploaded>
FROM imgstore.servers
WHERE name = #server_name)
This is the syntax for SQL Server, but I think it will work with MySQL as well.
Here's an article on INSERT ... SELECT Syntax
Please see my comment above regarding a potential data integrity issue. I am assuming that the name field in your server table has a unique constraint placed on it.
There are a couple of ways that you can approach this INSERT, and I'm sure that some are better than others. I make no claim that my way is the best way, but it should work. I don't know how you're writing this query, so I'm going to use #FieldValue to represent the variable input. My approach is to use a subquery in your insert statement to get the data that you require.
INSERT INTO images (field1, field2... s_id) VALUES ('#field1val', '#field2val'... (SELECT s_id FROM servers WHERE name='#nameval'));
(MySQL) I am trying to migrate a "subscription" table into 3 new tables: "product", "subscription", "actual" where "actual" is the name of the actual product, say, newsletter. The subscription has a FK reference to product and the actual has FK reference into subscription.
I know the INSERT INTO SELECT statement can copy data probably from many table to one; is there a statement to do the opposite, one to many tables for my case?
I'm not aware of an SQL statement that will do what you want. Just do several INSERT INTO SELECTs one after the other. It may be faster to do them one at a time anyway.
I think you can use three seperate insert into select statements. First you convert the product table, then the subscription where you can use an embedded select to find the id in the product table:
insert into subscription (some_column, FK_id,...)
select something, (select id from product where <your where clause>),...
and finally convert the actual table using an embedded select to get the id from the subscription table.
I'm using MySQL 4.1. Some tables have duplicates entries that go against the constraints.
When I try to group rows, MySQL doesn't recognise the rows as being similar.
Example:
Table A has a column "Name" with the Unique proprety.
The table contains one row with the name 'Hach?' and one row with the same name but a square at the end instead of the '?' (which I can't reproduce in this textfield)
A "Group by" on these 2 rows return 2 separate rows
This cause several problems including the fact that I can't export and reimport the database. On reimporting an error mentions that a Insert has failed because it violates a constraint.
In theory I could try to import, wait for the first error, fix the import script and the original DB, and repeat. In pratice, that would take forever.
Is there a way to list all the anomalies or force the database to recheck constraints (and list all the values/rows that go against them) ?
I can supply the .MYD file if it can be helpful.
To list all the anomalies:
SELECT name, count(*) FROM TableA GROUP BY name HAVING count(*) > 1;
There are a few ways to tackle deleting the dups and your path will depend heavily on the number of dups you have.
See this SO question for ways of removing those from your table.
Here is the solution I provided there:
-- Setup for example
create table people (fname varchar(10), lname varchar(10));
insert into people values ('Bob', 'Newhart');
insert into people values ('Bob', 'Newhart');
insert into people values ('Bill', 'Cosby');
insert into people values ('Jim', 'Gaffigan');
insert into people values ('Jim', 'Gaffigan');
insert into people values ('Adam', 'Sandler');
-- Show table with duplicates
select * from people;
-- Create table with one version of each duplicate record
create table dups as
select distinct fname, lname, count(*)
from people group by fname, lname
having count(*) > 1;
-- Delete all matching duplicate records
delete people from people inner join dups
on people.fname = dups.fname AND
people.lname = dups.lname;
-- Insert single record of each dup back into table
insert into people select fname, lname from dups;
-- Show Fixed table
select * from people;
Create a new table, select all rows and group by the unique key (in the example column name) and insert in the new table.
To find out what is that character, do the following query:
SELECT HEX(Name) FROM TableName WHERE Name LIKE 'Hach%'
You will se the ascii code of that 'square'.
If that character is 'x', you could update like this:(but if that column is Unique you will have some errors)
UPDATE TableName SET Name=TRIM(TRAILING 'x' FROM Name);
I'll assume this is a MySQL 4.1 random bug. Somes values are just changing on their own for no particular reason even if they violates some MySQL constraints. MySQL is simply ignoring those violations.
To solve my problem, I will write a prog that tries to resinsert every line of data in the same table (to be precise : another table with the same caracteristics) and log every instance of failures.
I will leave the incident open for a while in case someone gets the same problem and someone else finds a more practical solution.