I created Unique Compound Index:
Alter Table TableX Add Unique Index `UniqueRecord` (A,B,C,D)
The issue is that sometimes C can be NULL.
I noticed that
`Insert IGNORE`
Was still in some cases adding duplicate records and this turned out to be when those incoming records had C as NULL.
I tested the hypothesis that this was an issue by doing:
Select concat(A,B,C,D) as Index from TableA where C is NULL
And Index in each of those cases was in fact NULL. Once I remove the null field from the select:
Select concat(A,B,D) as Index from TableA where C is NULL
I get the expected string values vs nulls.
So the question is, other than doing an update like set C='' where C is NULL is there some way to set up the Index so that it works? I am loathe to simply make the Index A,B,D as that might introduce unwanted dupes when C in fact is not NULL.
Update:
I did try using IfNull in the Index creation but Mysql did not like that:
Alter Table TableA Add Unique Index UniqueLocator (A,B,IfNull(C,''),D
Mysql said:
[Err] 1064 - You have an error in your SQL syntax;
check the manual that corresponds to your MySQL server version
for the right syntax to use near 'C,''),D)' at line 1
Yes MySQL allows NULLs in unique indexes, which is the right thing to do. But you can define column C as NOT NULL if you don't like that.
MySQL -- but not all databases -- allow duplicate NULL values in unique indexes. I believe the ANSI standard is rather ambiguous on this point (or perhaps even contradictory). You basically have two choices.
The first is to define a default value for the column. This may not be appealing in terms of code, but it will at least generate an error on duplicate insert. For instance, if "C" is a foreign key reference to an auto-incremented id, then you might use -1 or 0 as the default value. If it is a date, you might use the zero date.
The other solution is a trigger, where you manually check for the duplicate values before doing an insert (or update).
Related
I have a MySQL insertion template:
INSERT INTO $0.$1 ($2$ColumnList)
VALUES ($2$ValueList) ON DUPLICATE KEY UPDATE
This is a MySQL command file (*.sqlcmd) which a program uses to randomly pass ColumnList and ValueList. So, I don't know in advance what are the values or the columns.
Now, I wish to make a conditional insertion to insert only if the columns are NULL or NOT NULL. Please note that I know I can write a MySQL statement like where mycolumn IS NULL or NOT NULL.
But are there any standard MySQL flags similar to ON DUPLICATE KEY UPDATE type, which I can specify once and for all, and that helps me to get rid of mentioning individual column names one at a time with IS NULL or IS NOT NULL?
No, there's no way to do this automatically. You need to name each column explicitly:
ON DUPLICATE KEY UPDATE col1 = IFNULL(VALUES(col1), col1), col2 = IFNULL(VALUES(col2), col2), ...
Whatever script is creating the variable $ColumnList could presumably generate this ON DUPLICATE KEY UPDATE clause from the same original data.
I have to migrate an old Paradox database to MySql. The Paradox database contains composite primary keys with null values (composite keys are composed by 3,4 or 5 fields and it could have 1 or 2 fields with a Null value). The problem is null values in pk are not allowed in MySql. It is impossible to replace the null values directly in Paradox (5 millions of lines in some tables), so how could we do ?
Thanks in advance for your solutions.
Assuming nulls in PK means they aren't needed to be unique, they could be any value, so assign an arbitrary value to nulls.
To do this, create a trigger on the MySQL table like this:
delimiter //
create trigger trig_pk_col_1 before insert on mytable
for each row
begin
set new.pk_col_1 = coalesce(new.pk_col_1, 999);
set new.pk_col_2 = coalesce(new.pk_col_2, 999);
-- etc for other
end;//
delimiter ;
I've chosen 999 as the arbitrary non-null value, but you can chose anything.
If the logic needs to be more sophisticated, you can code it as you like.
Also, I used coalesce() for brevity, but you can use the more verbose if instead if you prefer:
if new.pk_col_1 is null then
set new.pk_col_1 = 999;
end if;
-- etc
I think you have two choices in the MySQL DB,
(a) use a PK, which requires to replace null values / ignore such rows
(b) use a UNIQUE-constraint, which still allows null values, instead of a PK.
For option (a),
I'd suggest to disable the primary key constraint and import the data; then do all necessary corrections and reactivate the primary key. If you want to get rid of rows with null values in the potential PK columns, you could also make use of the IGNORE-keyword, which skips rows violating PK-constraints when inserting (cf. primary key and unique index constraints)
Option (b),
should allow to import the data as is. You can then do corrections or leave the data as is.
I have a simple MYSQL table with following columns:
first | second
first and second are integers. The primary key for the table is
PRIMARY KEY (`first`,`second`)
So this allows only a unique combination of values like:
first | second
1 | 2
2 | 1
But this key also accepts the same value for both columns. For example:
first | second
1 | 1
Is there a way to force both values to be different using MYSQL. I can do a check with PHP before inserting into the database but I'm wondering if there is a way in MYSQL to achieve it?
This restriction can't be enforced by a PRIMARY KEY or UNIQUE constraint.
Unfortunately, MySQL does not enforce CHECK CONSTRAINTS, which is what we would likely use in other databases.
To get MySQL to enforce a constraint like this, you would need to implement a BEFORE INSERT and a BEFORE UPDATE trigger.
The "trick" in the trigger body would be to detect this condition you want to restrict, e.g.
IF (NEW.first = NEW.second) THEN
And then have the trigger throw an error. In more recent versions of MySQL provide the SIGNAL statement for raising an exception. In older versions of MySQL, you'd run a statement that would throw an error (for example, performing a SELECT against a table name that is known not to exist.)
FOLLOWUP
The IF statement is valid only within the context of a MySQL stored program (for example, a PROCEDURE, FUNCTION, or TRIGGER).
To get this kind of restriction applied by an INSERT statement itself, without a constraint or trigger, we'd need to use the INSERT ... SELECT form of an INSERT statement.
For example:
INSERT INTO `mytable` (`first`, `second`)
SELECT t.first, t.second
FROM ( SELECT '1' AS `first, '1' AS `second`) t
WHERE t.first <> t.second
Since the SELECT statement returns no rows, no rows are inserted to the table.
Note that this approach applies the restriction only on this statement; This doesn't prevent some other session from performing an INSERT that doesn't enforce this restriction. To get this restriction enforced as a constraint "by the database", you'd need to implement a BEFORE INSERT and BEFORE UPDATE trigger I described earlier in the answer.
I've got an error on MySQL while trying to add a UNIQUE KEY. Here's what I'm trying to do. I've got a column called 'unique_id' which is VARCHAR(100). There are no indexes defined on the table. I'm getting this error:
#1062 - Duplicate entry '' for key 'unique_id'
When I try to add a UNIQUE key. Here is a screenshot of how I'm setting it up in phpMyAdmin:
Here is the MySQL query that's generate by phpMyAdmin:
ALTER TABLE `wind_archive` ADD `unique_id` VARCHAR( 100 ) NOT NULL FIRST ,
ADD UNIQUE (
`unique_id`
)
I've had this problem in the past and never resolved it so I just rebuilt the table from scratch. Unfortunately in this case I cannot do that as there are many entries in the table already. Thanks for your help!
The error says it all:
Duplicate entry ''
So run the following query:
SELECT unique_id,COUNT(unique_id)
FROM yourtblname
GROUP BY unique_id
HAVING COUNT(unique_id) >1
This query will also show you the problem
SELECT *
FROM yourtblname
WHERE unique_id=''
This will show you where there are values that have duplicates. You are trying to create a unique index on a field with duplicates. You will need to resolve the duplicate data first then add the index.
This is 3rd time i am looking for solution to this problem so for the reference I am posting the answer here.
Depending on the data we may use IGNORE keyword with Alter command. If IGNORE is specified, only the first row is used of rows with duplicates on a unique key, The other conflicting rows are deleted. Incorrect values are truncated to the closest matching acceptable value.
The IGNORE keyword extension to MySQL seems to have a bug in the InnoDB version on some version of MySQL.
You could always, convert to MyISAM, IGNORE-ADD the index and then convert back to InnoDB
ALTER TABLE table ENGINE MyISAM;
ALTER IGNORE TABLE table ADD UNIQUE INDEX dupidx (field);
ALTER TABLE table ENGINE InnoDB;
Note, if you have Foreign Key constraints this will not work, you will have to remove those first, and add them back later.
Make unique_id NULL from NOT NULL and it will solve your problem
select ID from wind_archive
where ID not in (select max(ID) from wind_archive group by unique_id)
and this is what you should remove from the table before you succesfully add the unique key.
this also works for adding unique key with 2 or more columns.
such as -
delete from wind_archive
where ID in (
select * from (select ID from wind_archive where ID not in (
select max(ID) from wind_archive group by lastName, firstName
) ORDER BY ID
) AS p
);
because of you write in your query, unique_id be NOT NULL and previous rows all of them are null and you want this column be unique, then after run this query, you have several rows with the same value it means this column is not unique, then you have to change unique_id NOT NULL to unique_id NULL in your query.
I was getting the same error (Duplicate entry '' for key 'unique_id') when trying to add a new column as unique "after" I had already created a table containing just names of museums. I wanted to go back and add a unique code for each museum name, with the intention of inserting the code values one at a time. Poor table planning on my part.
My solution was to add the new column without making it unique; then entered the data for each code one row at a time; and then changing the column structure to make it unique for future entries. Lucky there were only 10 rows.
I have a table with references from two of it's columns to two other tables PK's. Is there a way I can restrict both of those columns from having values set?
I only want one of them to have a value set, while the other is NULL
Favorites:
id
table_1_id
table_2_id
Table_1:
id
Table_2:
id
In SQL, you'd ideally handle this with a CHECK constraint.
In MySQL, there is not a direct mechanism to impose such a CHECK constraint. If you try to include one, the MySQL Reference Manual states (since it's part of the standard):
The CHECK clause is parsed but ignored by all storage engines.