I have to migrate an old Paradox database to MySql. The Paradox database contains composite primary keys with null values (composite keys are composed by 3,4 or 5 fields and it could have 1 or 2 fields with a Null value). The problem is null values in pk are not allowed in MySql. It is impossible to replace the null values directly in Paradox (5 millions of lines in some tables), so how could we do ?
Thanks in advance for your solutions.
Assuming nulls in PK means they aren't needed to be unique, they could be any value, so assign an arbitrary value to nulls.
To do this, create a trigger on the MySQL table like this:
delimiter //
create trigger trig_pk_col_1 before insert on mytable
for each row
begin
set new.pk_col_1 = coalesce(new.pk_col_1, 999);
set new.pk_col_2 = coalesce(new.pk_col_2, 999);
-- etc for other
end;//
delimiter ;
I've chosen 999 as the arbitrary non-null value, but you can chose anything.
If the logic needs to be more sophisticated, you can code it as you like.
Also, I used coalesce() for brevity, but you can use the more verbose if instead if you prefer:
if new.pk_col_1 is null then
set new.pk_col_1 = 999;
end if;
-- etc
I think you have two choices in the MySQL DB,
(a) use a PK, which requires to replace null values / ignore such rows
(b) use a UNIQUE-constraint, which still allows null values, instead of a PK.
For option (a),
I'd suggest to disable the primary key constraint and import the data; then do all necessary corrections and reactivate the primary key. If you want to get rid of rows with null values in the potential PK columns, you could also make use of the IGNORE-keyword, which skips rows violating PK-constraints when inserting (cf. primary key and unique index constraints)
Option (b),
should allow to import the data as is. You can then do corrections or leave the data as is.
Related
I have a table that has a primary key and for some reason i was advised not to use the AUTO_INCREMENT flag.
So every time i have to insert a new entry i search for the last value inserted (highest value) of the primary key.
Then i increment it by 1 to get a new ID.
Now the problem i face is, when inserting the first entry, there is no data in the table.
Can anyone suggest the optimal way to check
if data exists in table,
if not set id as 1 and insert new row,
else get the last id from table, increment it by 1 and then insert new row.
PS: New to mysql so having difficulty with its syntax.
Based on your statement 'i search for the last value inserted (highest value) of the primary key', I presume that you're currently doing something like this to get the maximum existing ID:
SELECT MAX(id_column) + 1 FROM my_table
If you have an empty table, this will of course return NULL. In that case, just handle the NULL using IFNULL to return 0 if there is no maximum value:
SELECT IFNULL(MAX(id_column), 0) + 1 FROM my_table
This will output 1 as the next identifier if the table has no rows.
I've given this more thought, and it turns out there is a way to generate a unique primary key without using Auto Increment or worrying about race conditions, so long as you are willing and able to use a 36 byte primary key (or, alternatively, a 128 bit binary).
The solution (at least as of MySQL 5.5) is the UUID, which stands for Universal Unique Identifier.
You would use it thus:
CREATE TABLE uu_table (
id VARCHAR(36) PRIMARY KEY,
name VARCHAR(50),
{other interesting columns}
)
Then insert new rows thus:
INSERT INTO uu_table VALUES (UUID(), 'Name of this Row', {other interesting values});
The UUID() function is guaranteed to generate a unique key 99.99{bunch more 9's}% of the time, even if generated on independent systems. That's its whole purpose, to be as unique as snowflake patterns, no matter where it is created.
There are pros and cons to this method. Best to read up on it here: https://dev.mysql.com/doc/refman/5.7/en/miscellaneous-functions.html#function_uuid
It is basically a 128 bit number, which you can save as a binary value after conversion from the 36 characters. I believe some versions of MySQL come with functions for that purpose. That would use less space in your database than 36 bytes, but I'll leave that as an exercise for the reader.
Currently, I have a mySQL table with columns that looks something like this:
run_date DATE
name VARCHAR(10)
load INTEGER
sys_time TIME
rec_time TIME
valid TINYINT
The column valid is essentially a valid bit, 1 if this row is the latest value for this (run_date,name) pair, and 0 if not. To make insertions simpler, I wrote a stored procedure that first runs an UPDATE table_name SET valid = 0 WHERE run_date = X AND name = Y command, then inserts the new row.
The table reads are in such a way that I usually use only the valid = 1 rows, but I can't discard the invalid rows. Obviously, this schema also has no primary key.
Is there a better way to structure this data or the valid bit, so that I can speed up both inserts and searches? A bunch of indexes on different orders of columns gets large.
In all of the suggestions below, get rid of valid and the UPDATE of it. That is not scalable.
Plan A: At SELECT time, use 'groupwise max' code to locate the latest run_date, hence the "valid" entry.
Plan B: Have two tables and change both when inserting: history, with PRIMARY KEY(name, run_date) and a simple INSERT statement; current, with PRIMARY KEY(name) and INSERT ... ON DUPLICATE KEY UPDATE. The "usual" SELECTs need only touch current.
Another issue: TIME is limited to 838:59:59 and is intended to mean 'time of day', not 'elapsed time'. For the latter, use INT UNSIGNED (or some variant of INT). For formatting, you can use sec_to_time(). For example sec_to_time(3601) -> 01:00:05.
I am using mysql, and am looking at a strange behavior.
Scenario :
I have a table having table_id as primary key, which is set to auto-increment.
table_id more_columns
1 some value
2 others
Now if i delete row 2, and insert one more row, the table_id becomes 3 (Expected is 2)
table_id more_columns
1 some value
3 recent
Why is it so? Here I am loosing some ids (I know they are not important). Please put some lights on this behavior
In auto-increment field If a row is deleted, the auto_increment column of that row will not be re-assigned.
Please see here for more information.
For reasons why auto-increment doesn't use deleted values you can refer here(mentioned in comments by #AaronBlenkush).
The auto_increment value is a counter stored internally for each table. The counter is only increased, never decreased.
Every syntactically correct INSERT statement fired against the table increments this counter, even when it is rolled back and also when you define an insert value for the primary key.
A MySQL auto_increment column maintains a number internally, and will always increment it, even after deletions. If you need to fill in an empty space, you have to handle it yourself in PHP, rather than use the auto_increment keyword in the table definition.
Rolling back to fill in empty row ids can cause all sorts of difficulty if you have foreign key relationships to maintain, and it really isn't advised.
The auto_increment can be reset using a SQL statement, but this is not advised because it will cause duplicate key errors.
-- Doing this will cause problems!
ALTER table AUTO_INCREMENT=12345;
EDIT
To enforce your foreign key relationships as described in the comments, you should add to your table definition:
FOREIGN KEY (friendid) REFERENCES registration_table (id) ON DELETE SET NULL;
Fill in the correct table and column names. Now, when a user is deleted from the registration, their friend association is nulled. If you need to reassociate with a different user, that has to be handled with PHP. mysql_insert_id() is no longer helpful.
If you need to find the highest numbered id still in the database after deletion to associate with friends, use the following.
SELECT MAX(id) FROM registration_table;
After delete write this query
ALTER TABLE tablename AUTO_INCREMENT = 1
I've got a bit of a stupid question. The thing is my program has to have the function to delete data from my database. Yay, not really the problem. But how can I delete data without the danger that others can see, that there has been something deleted.
User Table:
U_ID U_NAME
1 Chris
2 Peter
OTHER TABLE
ID TIMESTAMP FK_U_D
1 2012-12-01 1
2 2012-12-02 1
Sooooo the ID's are AUTO_INCREMENT, so if I delete one of them there's a gap. Furthermore, the timestamp is also bigger than the row before, so ascending.
I want to let the data with ID 1 disappear from the user's profile (U_ID 1).
If I delete it, there is a gap. If I just change the FK_U_ID to 2 (Peter) it's obvious, because when I insert data, there are 20 or 30 data rows with the same U_ID...so it's obvious that there has been a modification.
If I set the FK_U_ID NULL --> same sh** like when I change it to another U_ID.
Is there any solution to get this work? I know that if nobody but me has access to the database, it's just no problem. But just in case, if somebody controls my program it should not be obvious that there has been modifications.
So here we go.
For the ID gaps issue you can use GUIDs as #SLaks suggests, but then you can't use the native RDBMS auto_increment which means you have to create the GUID and insert it along with the rest of the record data upon creation. Of course, you don't really need the ID to be globally unique, you could just store a random string of 20 characters or something, but then you have to do a DB read to see if that ID is taken and repeat (recursively) that process until you find an unused ID... could be quite taxing.
It's not at all clear why you would want to "hide" evidence that a delete was performed. That sounds like a really bad idea. I'm not a fan of promulgating misinformation.
Two of the characteristics of an ideal primary key are:
- anonymous (be void of any useful information, doesn't matter what it's set to)
- immutable (once assigned, it will never be changed.)
But, if we set that whole discussion aside...
I can answer a slightly different question (an answer you might find helpful to your particular situation)
The only way to eliminate a "gap" in the values in a column with an AUTO_INCREMENT would be to change the column values from their current values to a contiguous sequence of new values. If there are any foreign keys that reference that column, the values in those columns would need to be updated as well, to preserve the relationship. That will likely leave the current auto_increment value of the table higher than the largest value of the id column, so I'd want to reset that as well, to avoid a "gap" on the next insert.
(I have done re-sequencing of auto_increment values in development and test environments, to "cleanup" lookup tables, and to move the id values of some tables to ranges that are distinct from ranges in other tables... that let's me test SQL to make sure the SQL join predicates aren't inadvertently referencing the wrong table, and returning rows that look correct by accident... those are some reasons I've done reassignment if auto_increment values)
Note that the database can "automagically" update foreign key values (for InnnoDB tables) when you change the primary key value, as long as the foreign key constraint is defined with ON UPDATE CASCADE, and FOREIGN_KEY_CHECKS is not disabled.
If there are no foreign keys to deal with, and assuming that all of the current values of id are positive integers, then I've been able to do something like this: (with appropriate backups in place, so I can recover if things don't work right)
UPDATE mytable t
JOIN (
SELECT s.id AS old_id
, #i := #i + 1 AS new_id
FROM mytable s
CROSS
JOIN (SELECT #i := 0) i
ORDER BY s.id
) c
ON t.id = c.old_id
SET t.id = c.new_id
WHERE t.id <> c.new_id
To reset the table AUTO_INCREMENT back down to the largest id value in the table:
ALTER TABLE mytable AUTO_INCREMENT = 1;
Typically, I will create a table and populate it from that query in the inline view (aliased as c) above. I can then use that table to update both foreign key columns and the primary key column, first disabling the FOREIGN_KEY_CHECKS and then re-enabling it. (In a concurrent environment, where other processes might be inserting/updating/deleting rows from one of the tables, I would of course first obtain an exclusive lock on all of the tables to be updated.)
Taking up again, the discussion I set aside earlier... this type of "administrative" function can be useful in a test environment, when setting up test cases. But it is NOT a function that is ever performed in a production environment, with live data.
I have a "tasks" table with a priority column, which has a unique constraint.
I'm trying to swap the priority value of two rows, but I keep violating the constraint. I saw this statement somewhere in a similar situation, but it wasn't with MySQL.
UPDATE tasks
SET priority =
CASE
WHEN priority=2 THEN 3
WHEN priority=3 THEN 2
END
WHERE priority IN (2,3);
This will lead to the error:
Error Code: 1062. Duplicate entry '3' for key 'priority_UNIQUE'
Is it possible to accomplish this in MySQL without using bogus values and multiple queries?
EDIT:
Here's the table structure:
CREATE TABLE `tasks` (
`id` int(11) NOT NULL,
`name` varchar(200) DEFAULT NULL,
`priority` varchar(45) DEFAULT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `priority_UNIQUE` (`priority`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8
Is it possible to accomplish this in MySQL without using bogus values and multiple queries?
No. (none that I can think of).
The problem is how MySQL processes updates. MySQL (in difference with other DBMS that implement UPDATE properly), processes updates in a broken manner. It enforces checking of UNIQUE (and other) constraints after every single row update and not - as it should be doing - after the whole UPDATE statement completes. That's why you don't have this issue with (most) other DBMS.
For some updates (like increasing all or some ids, id=id+1), this can be solved by using - another non-standard feature - an ORDER BY in the update.
For swapping the values from two rows, that trick can't help. You'll have to use NULL or a bogus value (that doesn't exist but is allowed in your column) and 2 or 3 statements.
You could also temporarily remove the unique constraint but I don't think that's a good idea really.
So, if the unique column is a signed integer and there are no negative values, you can use 2 statements wrapped up in a transaction:
START TRANSACTION ;
UPDATE tasks
SET priority =
CASE
WHEN priority = 2 THEN -3
WHEN priority = 3 THEN -2
END
WHERE priority IN (2,3) ;
UPDATE tasks
SET priority = - priority
WHERE priority IN (-2,-3) ;
COMMIT ;
I bumped into the same issue. Had tried every possible single-statement query using CASE WHEN and TRANSACTION - no luck whatsoever. I came up with three alternative solutions. You need to decide which one makes more sense for your situation.
In my case, I'm processing a reorganized collection (array) of small objects returned from the front-end, new order is unpredictable (this is not a swap-two-items deal), and, on top of everything, change of order (usually made in English version) must propagate to 15 other languages.
1st method: Completely DELETE existing records and repopulate entire collection using the new data. Obviously this can work only if you're receiving from the front-end everything that you need to restore what you just deleted.
2st method: This solution is similar to using bogus values. In my situation, my reordered collection also includes original item position before it moved. Also, I had to preserve original index value in some way while UPDATEs are running. The trick was to manipulate bit-15 of the index column which is UNSIGNED SMALLINT in my case. If you have (signed) INT/SMALLINT data type you can just invert the value of the index instead of bitwise operations.
First UPDATE must run only once per call. This query raises 15th bit of the current index fields (I have unsigned smallint). Previous 14 bits still reflect original index value which is never going to come close to 32K range.
UPDATE *table* SET `index`=(`index` | 32768) WHERE *condition*;
Then iterate your collection extracting original and new index values, and UPDATE each record individually.
foreach( ... ) {
UPDATE *table* SET `index`=$newIndex WHERE *same_condition* AND `index`=($originalIndex | 32768);
}
This last UPDATE must also run only once per call. This query clears 15th bit of the index fields effectively restoring original index value for records where it hasn't changed, if any.
UPDATE *table* SET `index`=(`index` & 32767) WHERE *same_condition* AND `index` > 32767;
Third method would be to move relevant records into temporary table that doesn't have a primary key, UPDATE all indexes, then move all records back to first table.
Bogus value option:
Okay, so my query is similar and I've found a way to update in "one" query. My id column is PRIMARY and position is part of a UNIQUE group. This is my original query that doesn't work for swapping:
INSERT INTO `table` (`id`, `position`)
VALUES (1, 2), (2, 1)
ON DUPLICATE KEY UPDATE `position` = VALUES(`position`);
.. but position is an unsigned integer and it's never 0, so I changed the query to the following:
INSERT INTO `table` (`id`, `position`)
VALUES (2, 0), (1, 2), (2, 1)
ON DUPLICATE KEY UPDATE `position` = VALUES(`position`);
.. and now it works! Apparently, MYSQL processes the values groups in order.
Perhaps this would work for you (not tested and I know almost nothing about MYSQL):
UPDATE tasks
SET priority =
CASE
WHEN priority=3 THEN 0
WHEN priority=2 THEN 3
WHEN priority=0 THEN 2
END
WHERE priority IN (2,3,0);
Good luck.
Had a similar problem.
I wanted to swap 2 id's that were unique AND was a FK from an other table.
The fastest solution for me to swap two unique entries was:
Create a ghost entry in my FK table.
Go back to my table where I want to switch the id's.
Turned of the FK Check SET FOREIGN_KEY_CHECKS=0;
Set my first(A) id to the ghost(X) fk (free's A)
Set my second (B) id to A (free's B)
Set A to B (free's X)
Delete ghost record and turn checks back on. SET FOREIGN_KEY_CHECKS=1;
Not sure if this would violate the constraints, but I have been trying to do something similar and eventually came up with this query by combining a few of the answers I found:
UPDATE tasks as T1,tasks as T2 SET T1.priority=T2.priority,T2.priority=T1.priority WHERE (T1.task_id,T2.task_id)=($T1_id, $T2_id)
The column I was swapping did not use a unique, so I am unsure if this will help...
you can achieve swapping your values with your above mentioned update statement, with a slight change in your key indexes.
CREATE TABLE `tasks` ( `id` int(11) NOT NULL, `name` varchar(200) DEFAULT NULL, `priority` varchar(45) DEFAULT NULL, PRIMARY KEY (`id`,`priority`) ) ENGINE=InnoDB DEFAULT CHARSET=utf8;
This will have a primary key index as a combination of id and priority. you cna then swap values.
UPDATE tasks
SET priority =
CASE
WHEN priority=2 THEN 3
WHEN priority=3 THEN 2
END
WHERE priority IN (2,3);
I dont see any need of user variables or temp variables here.
Hope this solves your issue :)