I have a column with data that exceeds MySQL's index length limit. Therefore, I can't use an unique key.
There's a solution here to the problem without using an unique key: MySQL: Insert record if not exists in table
However, in the comments, people are having issues with inserting the same value into multiple columns. In my case, a lot of my values are 0, so I'll get duplicate values very often.
I'm using Node and node-mysql to access the database. I'm thinking I can have a variable that keeps track of all values that are currently being inserted. Before inserting, I check if the value is currently being inserting. If so, I'll wait until it finishes inserting, then continue execution as if the value was originally inserted. However, I feel like this will be very error prone.
Here's part of my table schema:
CREATE TABLE `links` (
`id` int(10) UNSIGNED NOT NULL,
`url` varchar(2083) CHARACTER SET latin1 COLLATE latin1_general_cs NOT NULL,
`likes` int(10) UNSIGNED NOT NULL,
`tweets` int(10) UNSIGNED NOT NULL
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
ALTER TABLE `links`
ADD PRIMARY KEY (`id`),
ADD KEY `url` (`url`(50));
I cannot put an unique key on url because it can be 2083 bytes, which is over MySQL's key size limit. likes and tweets will often be 0, so the linked solution will not work.
Is there another possible solution?
If you phrase your INSERT in a certain way, you can make use of WHERE NOT EXISTS to check first if the URL does not exist before completing the insert:
INSERT INTO links (`url`, `likes`, `tweets`)
SELECT 'http://www.google.com', 10, 15 FROM DUAL
WHERE NOT EXISTS
(SELECT 1 FROM links WHERE url='http://www.google.com');
This assumes that the id column is a primary key/auto increment, and MySQL will automatically assign a value to it.
Related
Say I have this table:
CREATE TABLE `test` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`number` int(10) UNSIGNED NOT NULL,
PRIMARY KEY (id)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
Do I need some kind of lock if I insert data like this:
INSERT INTO test(number)
SELECT COALESCE(MAX(number), 0) + 1 FROM test;
In other words, if I have this statement executed in parallel multiple times, should I be worried that the same number could be inserted twice? I obviously will create a UNIQUE key (which will in fact be a composite key, that's why the classic AUTO INCREMENT feature does not fit my needs), but in that case should I be worry that a UNIQUE CONSTRAINT error might be thrown?
By default InnoDB uses auto-commit mode, so each query is a single transaction. So it will automatically perform the necessary locking to prevent duplication.
I got a MySQL database with some tables.
In one of these tables i want to insert by a SQL script some new rows.
Unfortunately i have to insert in two columns an empty string and the two columns are part of an unique key for that table.
So i tried to set UNIQUE_CHECKS before and after the insert, but i'm getting errors because of duplicate entries.
Here is the definition of the table:
CREATE TABLE `Table_A` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`name` varchar(100) NOT NULL,
`number` varchar(25) DEFAULT NULL,
`changedBy` varchar(150) DEFAULT NULL,
`changeDate` datetime NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
PRIMARY KEY (`id`),
UNIQUE KEY `name` (`name`,`number`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
And the INSERT statement which causes error:
SET UNIQUE_CHECKS = 0;
INSERT INTO `Table_A`
(`name`, `number`, `changedBy`, `changeDate`)
SELECT DISTINCT '', 'myUser', CURRENT_TIMESTAMP
FROM Table_A
AND id NOT IN
(
SELECT DISTINCT id
FROM Table_A
);
SET UNIQUE_CHECKS = 1;
As You can see, i'm using UNIQUE_CHECKS.
But as i said this doesn't work properly.
Any help or suggestion would be appreciated.
Patrick
Switching off Unique Keys for the insert operation doesn't indicate that it will check uniqueness only for the operations that happen after you switch it on again. It just means that database will not waste time to check the constraint during the time it is switch off but it will check the constraint when you switch it on again.
What it measn is that you nead to ensure that column has unique value in a columns with Unique Keys before you can turn it on. Which you don't do.
If you want to maintain Uniqueness somehow for new records you insert after some point in time you would need to create trigger and manually check the new records against already existing data. The same possibly goes for updates. But I don't recommend it - you should probably redesign data so either the Unique Key is not there or the data is truly unique for all the records there are and will be.
I've been getting this error from an insert on duplicate update query in MYSQL randomly every now and then.
Any idea what's going on? I can't seem to reproduce the error consistently it occurs sometimes and then sometimes not.
Here is the query in question:
INSERT INTO friendships (u_id_1,u_id_2,status) VALUES (?,?,'active') ON DUPLICATE KEY UPDATE id=LAST_INSERT_ID(id);
And the schema describing the table is:
DROP TABLE IF EXISTS `friendships`;
CREATE TABLE `friendships` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`u_id_1` int(11) NOT NULL,
`u_id_2` int(11) NOT NULL,
`status` enum('active','pending','rejected','blocked') DEFAULT 'pending' NOT NULL,
`initiatiator` enum('1','2','system') DEFAULT 'system' NOT NULL,
`terminator` enum('1','2','system') DEFAULT NULL,
`confirm_timestamp` timestamp DEFAULT NULL,
`created` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
PRIMARY KEY (`id`),
UNIQUE KEY (`u_id_1`,`u_id_2`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1;
Your ON DUPLICATE KEY UPDATE statement isn't helping you at all here.
You are taking the LAST_INSERT_ID, which is the auto inc of the last successfully inserted row, and trying to update the duplicated row with that id. This will always cause a duplicate primary (you're trying to change the id of some row to match the id of the last thing you added)
If your goal is to either
Insert a new row, or
Update an existing row with 'active'
Then
INSERT INTO friendships (u_id_1,u_id_2,status)
VALUES ( ? , ? ,'active')
ON DUPLICATE KEY UPDATE
status = 'active'; -- I changed this
A separate consideration is to check the source for duplicates. I had a simple audit table
INSERT INTO table
field1, field2, ... , field3
ON DUPLICATE KEY UPDATE row_id=row_id;
where field1 is an INDEX but not UNIQUE with row_ID as INTEGER UNSIGNED AUTO_INCREMENT PRIMARY KEY.
Ran for years, but an unexpected duplicate row triggered this error.
Fixed by de-duping the source.
Possibly a trivial point to many readers here, but it cost me some head-scratching (followed by a facepalm).
I need to add multiple records to a mysql database. I tried with multiple queries and its working fine, but not efficient. So I tried it with just one query like below,
INSERT INTO data (block, length, width, rows) VALUES
("BlockA", "200", "10", "20"),
("BlockB", "330", "8", "24"),
("BlockC", "430", "7", "36")
ON DUPLICATE KEY UPDATE
block=VALUES(block),
length=VALUES(length),
width=VALUES(width),
rows=VALUES(rows)
But it always update the table (columns are block_id, block, length, width, rows).
Should I do any changes on the query with adding block_id also. block_id is the primary key. Any help would be appreciated.
I've run your query without any problem, are you sure you don't have other keys defined with the data table ? And also make sure you have 'auto increment' set for the id field. without auto_increment, the query always update existing row
***** Updated **********
Sorry I've mistaken your questions. Yes, with only one auto_increment key, you query will always insert new rows instead of updating existing one ( because the primary key is the only way to detect 'existing' / duplication ), since the key is auto_increment, there's never a duplication if the primary key is not given in the insert query.
I think what you want to achieve is different, you might want to set up composite unique key on all fields (i.e. block, field, width, rows )
By the way, i've set up a SQL fiddle for you.
http://sqlfiddle.com/#!2/e7216/1
The syntax to add the unique key:
CREATE TABLE `data` (
`id` int(11) unsigned NOT NULL AUTO_INCREMENT,
`block` varchar(10) DEFAULT NULL,
`length` int(11) DEFAULT NULL,
`width` int(11) DEFAULT NULL,
`rows` int(11) DEFAULT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `uniqueme` (`block`,`length`,`width`,`rows`)
) ENGINE=InnoDB AUTO_INCREMENT=2 DEFAULT CHARSET=utf8;
I have a MySQL table that looks like this:
`id` int(10) unsigned NOT NULL AUTO_INCREMENT,
`company_id` int(8) unsigned NOT NULL,
`term_type` varchar(255) NOT NULL DEFAULT '',
`term` varchar(255) NOT NULL DEFAULT '',
I would like to be able to do this...
INSERT IGNORE INTO table ( company_id, term_type, term )
VALUES( a_company_id, 'a_term_type', 'a_term' )
... but I'd like the insert to be ignored when the same combination of company_id, term_type and term already exists. I am aware that if I have a unique index on a single field when I try to insert a duplicate value, the insert will be ignored. Is there a way to do the combo that I'm attempting? Could I use a multi-column index?
I'm trying to avoid doing a SELECT to check for this combination before every insert. As I'm processing hundreds of millions of rows of data into this table.
Maybe something like this:
ALTER TABLE table ADD UNIQUE (company_id, term_type,term);
If you use the IGNORE keyword, errors that occur while executing the INSERT statement are treated as warnings instead. For example, without IGNORE, a row that duplicates an existing UNIQUE index or PRIMARY KEY value in the table causes a duplicate-key error and the statement is aborted. With IGNORE, the row still is not inserted, but no error is issued.
So if you have a multicolumn primary key - it works.