I'm encountering an unexpected situation with inserting/querying particular records in the following table during periods of high contention. I believe there is a race condition in the database.
CREATE TABLE `business_objects` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`obj_id` varchar(255) DEFAULT NULL,
`obj_type` varchar(255) DEFAULT NULL,
`created_at` datetime(6) NOT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `index_business_objects_on_obj_type_and_obj_id`
(`obj_type`,`obj_id`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1;
The steps to reproduce are:
Check if record exists in table using this query
SELECT *
FROM business_objects
WHERE obj_type = 'Tip'
AND obj_id = '7616904'
If this query is null, attempt to create
INSERT INTO business_objects (obj_type, obj_id)
VALUES ('Tip', '7616904')
If another thread has already created a record with the same attributes this insert will fail and raise a uniqueness MySQL error. In this scenario, I catch the error and run the same query in step 1 to get the record.
SELECT *
FROM business_objects
WHERE obj_type = 'Tip'
AND obj_id = '7616904'
The query returns an empty result.
My expectation is that if the index uniqueness constraint is violated than the record should be committed to the table. What am I missing?
Related
So I'm trying to migrate a table from MySQL to MSSQL (sql server migration assistant MySQL), but I get this error:
Migrating data...
Analyzing metadata...
Preparing table testreportingdebug.testcase...
Preparing data migration package...
Starting data migration Engine
Starting data migration...
The data migration engine is migrating table '`testreportingdebug`.`testcase`': > [SwMetrics].[testreportingdebug].[testcase], 8855 rows total
Violation of UNIQUE KEY constraint 'testcase$Unique'. Cannot insert duplicate key in object 'testreportingdebug.testcase'. The duplicate key value is (<NULL>, <NULL>).
Errors: Violation of UNIQUE KEY constraint 'testcase$Unique'. Cannot insert duplicate key in object 'testreportingdebug.testcase'. The duplicate key value is (<NULL>, <NULL>).
Completing migration of table `testreportingdebug`.`testcase`...
Migration complete for table '`testreportingdebug`.`testcase`': > [SwMetrics].[testreportingdebug].[testcase], 0 rows migrated (Elapsed Time = 00:00:00:01:352).
Data migration operation has finished.
0 table(s) successfully migrated.
0 table(s) partially migrated.
1 table(s) failed to migrate.
I've just copied three rows from my table, and this is what they look like:
'1', 'Pump# TimeToService', NULL, NULL, 'A general test case comment ...', '0'
'2', 'Config.SlaveMinimumReplyDelay', NULL, NULL, NULL, '0'
'3', 'Config.RESERVED', NULL, NULL, NULL, '0'
If you are wondering how the colons in the MySQL table is setup, here you go:
Is is because right, left and comment can be null?
DDL of table
CREATE TABLE `testcase` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`TestCaseName` varchar(150) DEFAULT NULL,
`Left` int(11) DEFAULT NULL,
`Right` int(11) DEFAULT NULL,
`Comment` text,
`Hidden` tinyint(4) NOT NULL DEFAULT '0',
PRIMARY KEY (`id`),
UNIQUE KEY `Unique` (`Left`,`Right`)
) ENGINE=InnoDB AUTO_INCREMENT=10580 DEFAULT CHARSET=utf8
Had to remove the Unique part, since their are only NULL.
ALTER TABLE `testreportingdebug`.`testcase`
DROP INDEX `Unique`;
If you want the strict equivalent in SQL Server of your MySQL table you must create it like this :
CREATE TABLE testcase (
id int NOT NULL IDENTITY PRIMARY KEY,
TestCaseName varchar(150),
[Left] int,
[Right] int,
Comment VARCHAR(max),
[Hidden] tinyint DEFAULT 0,
);
CREATE UNIQUE INDEX X_testcase_right_left
ON testcase ([Left], [Right])
WHERE [Left] IS NOT NULL
AND [Right] IS NOT NULL;
By the way, column names "Right", "left", "hidden" are SQL / MS SQL Server reserved words and should not be used at anytime for SQL identifiers (table name, colum name, proc name...)
The complete list can be obtain here
I'm running MySQL 5.5 and found behaviour I didn't know of before.
Given this create:
CREATE TABLE `test` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`name` varchar(128) DEFAULT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `name_UQ` (`name`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
With these inserts:
insert into test (name) values ('b');
insert into test (name) values ('a');
And this select:
select * from test;
MySQL does something I wasn't aware of:
2 a
1 b
It sorts automatically.
Given a table with one extra, non-unique column:
CREATE TABLE `test` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`name` varchar(128) DEFAULT NULL,
`other_column` varchar(128) DEFAULT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `name_UQ` (`name`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
And the same inserts (see above), the select (see above) gives this result:
1 b NULL
2 a NULL
Which is kind of expected.
Where is the behaviour of the first query (SQL Fiddle) documented? I'd like to see more of these peculiar things.
MySQL does not sort result sets automatically. The ordering of a result set is indeterminate unless the query specifies an order by clause.
You should never rely on any sort of "implicit" ordering. Just because you see it in 1 (or 100 queries). In fact, without an order by, the same query can return results in different orders on subsequent runs (although I'll admit that this regularly occurs in other database, it is unlikely in MySQL).
Instead, add the ORDER BY. Ordering by a primary key is remarkably efficient, so you don't have to worry about performance.
I am in a problematic situation and found dozens of questions on same topic, but may b i am not able to understand those solutions as per my issue.
I have a system built in Codeigniter, and it does the following
codeigniter->start_transaction()
UPDATE T SET A = 1, MODIFIED = NOW()
WHERE PK IN
( SELECT PK FROM
(SELECT PK, LAST_INSERT_ID(PK) FROM T
where FK = 31 AND A=0 AND R=1 AND R_FK = 21
AND DEAD = 0 LIMIT 0,1) AS TBL1
) and A=0 AND R = 1 AND R_FK = 21 AND DEAD = 0
-- what this query does is , it takes a row dynamically which is not dead yet,
--and not assigned and it's linked to 21 id (R_FK) from R table,
-- when finds the row, update it to be marked as assigned (A=1).
-- PK = LAST_INSERT_ID(PK) ensures that last_insert_id is updated with this row id, so i can retrieve it from PHP
GOTO MODULE B
MODULE B {
INSERT INTO T(A,B,C,D,E,F,R,FK,R_FK,DEAD,MODIFIED) VALUES(ALL VALUES)
-- this line gives me lock wait timeout exceeded.
}
MySQL version is 5.1.63-community-log
Table T is an INNODB table and has only one normal type index on FK field, and no foreign key constraints are there. PrimaryKey (PK) field is an auto_increment field.
I get lock wait timeout in the above case , and that is due to first transactional update holding lock on table, how can i avoid lock on table with that update query ,while using transactions, I cannot commit the transaction until i receive response from MODULE B .
I don't have much detailed knowledge about DB and structural things, so please bear with me if i said something not making sense.
--UPDATE--
-- TABLE T Structure
CREATE TABLE `T` (
`PK` int(11) NOT NULL AUTO_INCREMENT,
`FK` int(11) DEFAULT NULL,
`P` varchar(1024) DEFAULT NULL,
`DEAD` tinyint(1) NOT NULL DEFAULT '0',
`A` tinyint(1) NOT NULL DEFAULT '0',
`MODIFIED` datetime DEFAULT NULL,
`R` tinyint(4) NOT NULL DEFAULT '0',
`R_FK` int(11) NOT NULL DEFAULT '0',
PRIMARY KEY (`PK`),
KEY `FK_REFERENCE_54` (`FK`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
-- Indexes Information
SHOW INDEX FROM T;
1- Field FK, Cardinality 65 , NULL => Yes , Index_Type => BTRee
2- Field PK, Cardinality 11153, Index_Type => BTRee
I've been getting this error from an insert on duplicate update query in MYSQL randomly every now and then.
Any idea what's going on? I can't seem to reproduce the error consistently it occurs sometimes and then sometimes not.
Here is the query in question:
INSERT INTO friendships (u_id_1,u_id_2,status) VALUES (?,?,'active') ON DUPLICATE KEY UPDATE id=LAST_INSERT_ID(id);
And the schema describing the table is:
DROP TABLE IF EXISTS `friendships`;
CREATE TABLE `friendships` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`u_id_1` int(11) NOT NULL,
`u_id_2` int(11) NOT NULL,
`status` enum('active','pending','rejected','blocked') DEFAULT 'pending' NOT NULL,
`initiatiator` enum('1','2','system') DEFAULT 'system' NOT NULL,
`terminator` enum('1','2','system') DEFAULT NULL,
`confirm_timestamp` timestamp DEFAULT NULL,
`created` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
PRIMARY KEY (`id`),
UNIQUE KEY (`u_id_1`,`u_id_2`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1;
Your ON DUPLICATE KEY UPDATE statement isn't helping you at all here.
You are taking the LAST_INSERT_ID, which is the auto inc of the last successfully inserted row, and trying to update the duplicated row with that id. This will always cause a duplicate primary (you're trying to change the id of some row to match the id of the last thing you added)
If your goal is to either
Insert a new row, or
Update an existing row with 'active'
Then
INSERT INTO friendships (u_id_1,u_id_2,status)
VALUES ( ? , ? ,'active')
ON DUPLICATE KEY UPDATE
status = 'active'; -- I changed this
A separate consideration is to check the source for duplicates. I had a simple audit table
INSERT INTO table
field1, field2, ... , field3
ON DUPLICATE KEY UPDATE row_id=row_id;
where field1 is an INDEX but not UNIQUE with row_ID as INTEGER UNSIGNED AUTO_INCREMENT PRIMARY KEY.
Ran for years, but an unexpected duplicate row triggered this error.
Fixed by de-duping the source.
Possibly a trivial point to many readers here, but it cost me some head-scratching (followed by a facepalm).
I have a MySQL query as follows:
SELECT KeywordText, SUM(Frequency) AS Frequency FROM Keyword, Keyword_Polling_Frequency_Index
WHERE Keyword.KeywordText
IN ('deal', 'obama' and other keywords...)
AND RSSFeedNo IN (106, 107 and other RSS feeds)
AND PollingDateTime
BETWEEN '2011-10-28 13:00:00' AND '2011-10-28 13:59:00'
AND Keyword.KeywordNo = Keyword_Polling_Frequency_Index.KeywordNo
GROUP BY Keyword.KeywordText
ORDER BY Keyword.KeywordText ASC
The query is used by an hourly batch program which involves two tables and is meant to get the frequencies of a list of keywords from a list of RSS feeds for a given hour. The Keyword_Polling_Frequency_Index table has a composite primary key of KeywordNo, RSSFeedNo and PollingDateTime. The query joins this table to the Keyword table which contains the KeywordText. column keywordText has a MySQL MyISAM full text index.
In testing this was found to perform satisfactorily but has now started running very slowly and affects the interactive speed of pages of the application. When I check the MySQL logs, I find that MySQL is creating temporary tables.
So, my question is, given that this query has to handle dozens of keywords in dozens of RSS feeds to calculate the frequencies, can anyone suggest an optimisation?
I have thought of breaking the query up by keyword but am not convinced of the practicality of this.
Can anyone help?
I am using MySQL Community Edition 5.X and an EXTENDED EXPLAIN of a version of this query is shown above.
SQL for the tables is as follows:
CREATE TABLE `keyword` (
`KeywordNo` int(10) unsigned NOT NULL AUTO_INCREMENT,
`KeywordText` varchar(64) NOT NULL,
`UserOriginated` enum('TRUE','FALSE') NOT NULL,
`Active` enum('TRUE','FALSE') NOT NULL,
`UserNo` varchar(50) NOT NULL,
`StopWord` enum('TRUE','FALSE') NOT NULL,
`CreatedDate` date NOT NULL,
`CreatedTime` time NOT NULL,
PRIMARY KEY (`KeywordNo`),
FULLTEXT KEY `KEYWORDTEXT` (`KeywordText`)
) ENGINE=MyISAM AUTO_INCREMENT=44047 DEFAULT CHARSET=latin1$$
CREATE TABLE `keyword_polling_frequency_index` (
`KeywordNo` int(10) unsigned NOT NULL,
`RSSFeedNo` int(10) unsigned NOT NULL,
`PollingDateTime` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
`Frequency` int(10) NOT NULL,
`Active` enum('TRUE','FALSE') NOT NULL,
`UserNo` varchar(50) NOT NULL,
PRIMARY KEY (`KeywordNo`,`RSSFeedNo`,`PollingDateTime`),
KEY `FK_keyword_polling_frequency_index_1` (`UserNo`),
CONSTRAINT `FK_keyword_polling_frequency_index_1` FOREIGN KEY (`UserNo`) REFERENCES `user` (`UserNo`) ON DELETE CASCADE ON UPDATE CASCADE
) ENGINE=InnoDB DEFAULT CHARSET=latin1$$
As mentioned previously, add an index to the PollingDateTime field in the order mentioned as well. This is my suggestion:
SELECT
K.KeywordText,
SUM(F.Frequency) AS Frequency
FROM
Keyword K, Keyword_Polling_Frequency_Index F
WHERE
EXISTS
(
SELECT 1
FROM Keyword K1
WHERE
MATCH K1.KeywordText AGAINST ('deal obama "another keyword" yetanother' IN BOOLEAN MODE)
AND K1.KeywordNo = K.KeywordNo
)
AND K.KeywordNo = F.KeywordNo
AND F.PollingDateTime BETWEEN '2011-10-28 13:00:00' AND '2011-10-28 13:59:00'
AND F.RSSFeedNo IN (106, 107, 110)
GROUP BY K.KeywordText
ORDER BY K.KeywordText ASC
This will probably reduce the number of records for the comparison (SQL inside-out parsing) instead of directly matching two tables (N x N).
If you don't have any indexes you should create relevant indexes.
The minimum index is on keyword_polling_frequency_index.PollingDateTime