I'm currently trying to create a table of theater locations that only has three locations. I imported denormalized data that I tried to normalize with this statement:
insert into theater(`name`, email, address, phone)
select distinct theater, theater_email, theater_address, theater_phone
from denormalized_tickets;
when I comment out the first line and run it I get the result I'm looking for.
When I write a query to see the theater table select * from theater;, it returns the theaters duplicated each 12 times.
How should I solve this? Is there anything I'm overlooking?
As discussed in the comments above, INSERT creates new rows each time you execute it. If you do that multiple times, you may add more rows every time.
Vasya recommended creating a UNIQUE index to block new rows from being created with the same values. This may or may not be appropriate for a given table. For instance, what if you want to allow multiple rows to have the same values?
Another thing you might like to read about is MySQL's REPLACE statement. The syntax is similar to INSERT, but if there's a duplicate in column(s) of a primary key or unique key, it first deletes the old row and then inserts the new row. But this won't help if you don't have the unique key defined, because how would MySQL know it's a conflict?
Related
If two individual records are added to a database such as two patients being added to the table 'Paitents' and when they are added the Primary_Key such as Paitient_ID is created automatically and given to each new account.
.(Auto Incremented)
That bit is quite straight forward and understand I can just use an 'INSERT INTO SONGS' statement.
But what if the two patients are related and I have another table called "Relations"
Where by I need it to pull in the two Paitent_ID's and create a relation from the same insert query. Can this be done?
ID generated by MySQL automatically in auto_increment column can be obtained using the LAST_INSERT_ID() function. Languages/libraries often offer a function for this (e.g. in PHP PDO you call PDO::lastInsertId() instead of making another query).
How to solve this exactly depends on how you are inserting the values into a database, but the basics can be:
Insert patient one
Get the ID
Insert patient two
Get the ID
Create the relation
I'm trying to filter rows from the MySQL table where all the $_POST data is stored from an online form. Sometimes the user's internet connection stalls or the browser screws up, and the new page after form submission is not displayed (though the INSERT worked and the table row was created). They then hit refresh, and submit their form twice, creating a duplicate row (except for the timestamp and autoincrement id columns).
I'd like to select unique form submissions. This has to be a really common task, but I can't seem to find something that lets me call with DISTINCT applying to every column except the timestamp and id in a succinct way (sort of like SELECT id, timestamp, DISTINCT everything_else FROM table;. At the moment, I can do:
CREATE TEMPORARY TABLE IF NOT EXISTS temp1 AS (
SELECT DISTINCT everything,except,id,and,timestamp
FROM table1
);
SELECT * FROM table1 LEFT OUTER JOIN temp1
ON table1.everything = temp1.everything
...
;
My table has 20k rows with about 25 columns (classification features for a machine learning exercise). This query takes forever (as I presume it traverses the 20k rows 20K times?) I've never even let it run to completion. What's the standard practice way to do this?
Note: This question suggests add an index to the relevant columns, but there can be max 16 key parts to an index. Should I just choose the most likely unique ones? I can find about 700 duplicates in 2 seconds this way, but I can't be sure of not throwing away a unique row because I also have to ignore some columns when specifying the index.
If you have a UNIQUE key (other than an AUTO_INCREMENT), simply use INSERT IGNORE ... to silently avoid duplicate rows. If you don't have a UNIQUE key, do you never need to find a row again?
If you have already allowed duplicates and you need to get rid of them, that is a different question.
I would try to eliminate the problem in the first place. There are techniques to eliminate this issue. The first one on my mind is that you could generate a random string and store it in both the session and as a hidden field in the form. This random string should be generated each time the form is displayed. When the user submits the form you need to check that the session key and the input key matches. Make sure to generate a different key on each request. Thus when a user refreshes the page he will submit an old key and it will not match.
Another solution could be that if this data should always be unique in the database check if there is that exact data in the database first before inserting. And if the data is unique by lets say the email address you can create a unique key index. Therefore that field will have to be unique in the table.
This has been discussed before, however I cannot understand the answers I have found.
Essentially I have a table with three columns memo, user and keyid (the last one is primary and AUTO_INC). I insert a pair of values (memo and user). But if I try to insert that same pair again it should not happen.
From what I found out, the methods to do this all depend on a unique key (which I've got, in keyid) but what I don't understand is that you still need to do a second query just to get the keyid of the existing couple (or get nothing, in which case you go ahead with the insertion).
Is there any way to do all of this in a single query? Or am I understanding what I've read (using REPLACE or IGNORE) wrong?
You need to set a UNIQUE KEY on user + memo,
ALTER TABLE mytable
ADD CONSTRAINT unique_user_memo UNIQUE (memo,user)
and then using INSERT IGNORE or REPLACE according to your needs when inserting. Your current unique key is the primary key, that is all well and good, but you need a 2nd one in order to not allow the insertion of duplicate data. If you do not create a new unique key on the two columns together, then you'll need to do a SELECT query before every insert to check if the pair already exists.
I have 2 equal databases (A and B) with one table each running in separate offline machines.
Every day I export their data (as csv) and "merge" it into a 3rd database (C). I first process A, then B (I insert the content from A to C, then the contents from B to C)
Now, it could happen that I get duplicate rows. I consider a duplicate if some field, for example "mail" already exists. I don't care if the rest of the fields are the same.
How can I insert A and B into C excluding those rows that are duplicates?
Thanks in advance!
Easiest solution should be to create a unique index on the columns in question and run the second insert as INSERT IGNORE
Personally I use the ON DUPLICATE KEY UPDATE as using INSERT IGNORE causes any errors to be thrown as warnings.
This may have some side effects and may result in behavior you may not expect. See this post for details on some of the side effects.
If you end up using the ON DUPLICATE KEY UPDATE syntax, it will also provide a means of changing your logic to update specific fields with new data should business requirements change.
For instance, you can tally how many times a duplicate record was inserted by saying ON DUPLICATE KEY UPDATE quantity = quantity+1.
The post referenced above has a ton more information.
I read this question, which highlights a solution to conditionally insert values into a table if they don't already exist. My question: is it possible to conditionally insert multiple values at once.
For instance, say I have a table that just contains user names (this is a pointless table, but let's keep it simple). The table's contents look like this:
matthew 20
mark 24
luke 25
john 56
buddy 68
A user enters jimmy 34, mark 25 and bobby 54 in a web form and submits, and I'd like to check whether those three values exist in the table already and insert the ones that don't in one statement. Yes, for this example, I'm assuming names are unique.
Here is a paraphrase of the code snippet from the question I linked to, adapted to this example:
INSERT INTO users(name)
SELECT 'jimmy'
FROM dual
WHERE NOT EXISTS (SELECT * FROM users
WHERE user = 'jimmy')
How can I adapt this for multiple values being inserted at once? It's also important that the solution work independently of the number of values entered. In my example, I give three (jimmy, mark and bobby) but there may only be one or there may be 20.
Second question: is this wise? I know that reducing the number of queries is desirable but is it worth it here? Should I just set up a for loop and loop through, alternately checking if a value exists and inserting if it doesn't?
Thanks for any help.
Sorry I don't have code that I've tried myself to show, I'm not even sure what to try here.
Update: added an extra column to the table. I wanted to keep things simple but need two columns to illustrate the fact that deleting a row and then inserting or updating are not what I want as they would favor the user's input over what is already in the table.
See INSERT IGNORE combined with a UNIQUE KEY on the field in question.
When a unique key constraint fails the whole row is silently ignored.
ALTER TABLE users ADD UNIQUE KEY `name` (`name`);
INSERT IGNORE INTO `users` (`name`, `age`)
VALUES ("jimmy", 22), ("bob", 45),
("luke", 300), ("john", 456);
Note: this will also suppress errors when datatypes mismatch and accurate conversion is impossible. (e.g. DECIMAL vs INT) MySQL will continue using the nearest result possible. (e.g. INT) You should ensure only pre-validated data is inserted with such statement.
I am not big on programming(especially on php), but can you parse the string into separate names and then do the loop?
something like (hopefuly the syntax is somewhat correct)
$names = split(" ", $user_names);
for ($i=0; $i<count($names); i++)
{
//sql query check - add new name function
function_sql_query($names[$i]);
}
In addition, you can add rules to your SQLDBMS to disallow dublicates in this attribute and add only unique records; this process will be maintained by DBMS
You can use MySQL's REPLACE
http://dev.mysql.com/doc/refman/5.0/en/replace.html
OR INSERT ... ON DUPLICATE KEY UPDATE
INSERT INTO table (a,b,c) VALUES (1,2,3)
ON DUPLICATE KEY UPDATE c=c;