I have a MySql table created like this:
CREATE TABLE `sourcelinks` (
`idSourceLinks` int(10) unsigned NOT NULL AUTO_INCREMENT,
`SrcId` int(11) NOT NULL DEFAULT '-1',
`LinkId` int(11) DEFAULT '-1',
`ImageId` int(11) DEFAULT '-1',
`DownloadId` int(11) DEFAULT '-1',
`VideoId` int(11) DEFAULT '-1',
PRIMARY KEY (`idSourceLinks`),
UNIQUE KEY `idSourceLinks_UNIQUE` (`idSourceLinks`),
UNIQUE KEY `UniqueCombination` (`SrcId`,`LinkId`,`ImageId`,`DownloadId`,`VideoId`)
) ENGINE=InnoDB AUTO_INCREMENT=491 DEFAULT CHARSET=latin1;
As you can see I have a combined UNIQUE INDEX over SrcId, LinkId, ImageId, DownloadId, and VideoId. Now my understanding is that if SrcId and LinkId have the same values as another row then a DUPLICATE INSERT exception would be thrown. That would be the same as SrcId and ImageId, or SrcId and DownloadId etc.
QUESTIONS:
So, why does it not actually work? I am getting multiple columns with the same values, that is multiple SrcId =1 and LinkId = 1 with the other columns in the index being null?
And how do i fix it so that each row can only have unique values attached for the columns
A unique index on (SrcId, LinkId, ImageId, DownloadId, VideoId) only means that a given combination of values of all five of these columns must be unique. It does not that more than one record cannot have the same values for SrcId and LinkId. If you require that, the unique index should be on (SrcId, LinkId). If this were the original intention of the five column unique index, then perhaps remove it and replace it with the two column version.
Related
i want to log timestamp, parameter_id and value into my db, using the timestamp and the parameter_id as the primary, unique key.
data_togo, CREATE TABLE 'data_togo' (
'id_para' int(10) unsigned NOT NULL DEFAULT '0',
't_ns' bigint(20) unsigned NOT NULL DEFAULT '0',
'id_inst' smallint(6) NOT NULL DEFAULT '1',
'value' varchar(255) NOT NULL DEFAULT '',
'isanchor' tinyint(4) unsigned NOT NULL DEFAULT '0',
PRIMARY KEY ('t_ns','id_para')
) ENGINE=InnoDB DEFAULT CHARSET=utf8
However in a very rare condition the case can happen where there are duplicate primary keys with different values.
INSERT INTO data_togo (id_para, t_ns, id_inst, value, is_anchor) VALUES ...
Is it possible to keep both rows and update the second timestamp (nanoseconds, so that would not matter) to +1?
EDIT: Problem: currently there is a bug where i get the data from, the timestamp is rounded to milliseconds with microseconds resolution and then brought into nanoseconds format. hence, if it goes badly both the DIFFERENT timestamps are rounded to the same value, which leads to a duplicate primary key.
I've found a solution that works for two times the same timestamp:
Transfer the whole thing into a stored procedure
If the row exists, insert it with the following statement:
INSERT INTO edl_dw.data_togo (id_para, t_ns, id_inst, value, isanchor)
VALUES(#id_para, #t_ns, #id_inst, #value, #anchor)
ON DUPLICATE KEY UPDATE t_ns= #t_ns-1;
Then insert another row into the table with the same content, but with insert ignore
INSERT IGNORE edl_dw.data_togo (id_para, t_ns, id_inst, value, isanchor)
VALUES(#id_para, #t_ns, #id_inst, #value, #anchor);
If the row already existed, the old row timestamp will be set 1 ns back, but the content is kept. Then the new row gets inserted after that.
If the row didn't exist, the row get's inserted in the first statement, and the second insert gets ignored because the primary key already exists.
I have simple categories table. Category can have parent category (par_cat column) or null if it is main category and with the same parent category there shouldn't be 2 or more categories with the same name or url.
Code for this table:
CREATE TABLE IF NOT EXISTS `categories` (
`id` int(10) unsigned NOT NULL,
`par_cat` int(10) unsigned DEFAULT NULL,
`lang` varchar(2) COLLATE utf8_unicode_ci NOT NULL DEFAULT 'pl',
`name` varchar(100) COLLATE utf8_unicode_ci NOT NULL,
`url` varchar(120) COLLATE utf8_unicode_ci NOT NULL,
`active` tinyint(3) unsigned NOT NULL DEFAULT '1',
`accepted` tinyint(3) unsigned NOT NULL DEFAULT '1',
`priority` int(10) unsigned NOT NULL DEFAULT '1000',
`entries` int(10) unsigned NOT NULL DEFAULT '0',
`created_at` timestamp NOT NULL DEFAULT '0000-00-00 00:00:00',
`updated_at` timestamp NOT NULL DEFAULT '0000-00-00 00:00:00'
) ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci AUTO_INCREMENT=3 ;
ALTER TABLE `categories`
ADD PRIMARY KEY (`id`),
ADD UNIQUE KEY `categories_name_par_cat_unique` (`name`,`par_cat`),
ADD UNIQUE KEY `categories_url_par_cat_unique` (`url`,`par_cat`),
ADD KEY `categories_par_cat_foreign` (`par_cat`);
ALTER TABLE `categories`
MODIFY `id` int(10) unsigned NOT NULL AUTO_INCREMENT,AUTO_INCREMENT=3;
ALTER TABLE `categories`ADD CONSTRAINT `categories_par_cat_foreign`
FOREIGN KEY (`par_cat`) REFERENCES `categories` (`id`);
The problem is that even if I have unique keys it doesn't work. If I try to insert into database 2 categories that have par_cat set to null and same name and url, those 2 categories can be inserted into database without a problem (and they shouldn't). However if I select for those categories other par_cat (for example 1 assuming category with id 1 exists), only first record will be inserted (and that's desired behaviour).
Question - how to handle this case? I read that:
A UNIQUE index creates a constraint such that all values in the index
must be distinct. An error occurs if you try to add a new row with a
key value that matches an existing row. This constraint does not apply
to NULL values except for the BDB storage engine. For other engines, a
UNIQUE index permits multiple NULL values for columns that can contain
NULL. If you specify a prefix value for a column in a UNIQUE index,
the column values must be unique within the prefix.
however if I have unique on multiple columns I expected it's not the case (only par_cat can be null, name and url cannot be null). Because par_cat references to id of the same table but some categories don't have parent category it should allow null values.
This works as defined by the SQL standard. NULL means unknown. If you have two records of par_cat = NULL and name = 'X', then the two NULLs are not regarded to hold the same value. Thus they don't violate the unique key constraint. (Well, one could argue that the NULLs still might mean the same value, but applying this rule would make working with unique indexes and nullable fields almost impossible, for NULL could as well mean 1, 2 or whatever other value. So they did well to define it such as they did in my opinion.)
As MySQL does not support functional indexes where you could have an index on ISNULL(par_cat,-1), name, your only option is to make par_cat a NOT NULL column with 0 or -1 or whatever for "no parent", if you want your constraints to work.
I see that this was asked in 2014.
However it is often requested from MySQL: https://bugs.mysql.com/bug.php?id=8173 and https://bugs.mysql.com/bug.php?id=17825 for example.
People can click on affects me to try and get attention from MySQL.
Since MySQL 5.7 we can now use the following workaround:
ALTER TABLE categories
ADD generated_par_cat INT UNSIGNED AS (ifNull(par_cat, 0)) NOT NULL,
ADD UNIQUE INDEX categories_name_generated_par_cat (name, generated_par_cat),
ADD UNIQUE INDEX categories_url_generated_par_cat (url, generated_par_cat);
The generated_par_cat is a virtual generated column, so it has no storage space. When a user inserts (or updates) then the unique indexes cause the value of generated_par_cat to be generated on the fly which is a very quick operation.
Just in case you come from Laravel...
This is Laravel's Migration version for Virtual Column to workaround the UNIQUE issue when one of the columns is NULL in value
$table->integer('generated_par_cat')->virtualAs('ifNull(par_cat, 0)');
$table->unique(['name', 'generated_par_cat'], 'name_par_cat_unique');
I have a table with 8 columns, as shown below in the create statement.
Rows have to be unique, that is, no two rows can have the exact same value in each column. To this end I defined each column to be a Primary Key.
However, performing a select as show below takes extremely long as, i suppose, MySQL will have to scan each row to find results. As the table is pretty large, this takes a lot of time.
Do you have any suggestion how I could increase performance?
EDIT create statement:
CREATE TABLE `volatilities` (
`instrument` varchar(45) NOT NULL DEFAULT '',
`baseCurrencyId` int(11) NOT NULL,
`tenor` varchar(45) NOT NULL DEFAULT '',
`tenorUnderlying` varchar(45) NOT NULL DEFAULT '',
`strike` double NOT NULL DEFAULT '0',
`evalDate` date NOT NULL DEFAULT '0000-00-00',
`volatility` double NOT NULL DEFAULT '0',
`underlying` varchar(45) NOT NULL DEFAULT '',
PRIMARY KEY (`instrument`,`baseCurrencyId`,`tenor`,`tenorUnderlying`,`strike`,`evalDate`,`volatility`,`underlying`)) ENGINE=InnoDB DEFAULT CHARSET=utf8
Select statement:
SELECT evalDate,
max(case when strike = 0.25 then volatility end) as '0.25'
FROM niels_testdb.volatilities
WHERE
instrument = 'Swaption' and tenor = '3M'
and tenorUnderlying = '3M' and strike = 0.25
GROUP BY
evalDate
One of your requirements is that all the rows need to have unique values. So that is why you created the table with composite primary keys for all columns. But your table WOULD allow duplicated values for every column, as long as the rows themselves were unique.
Take a look at this sql fiddler post: http://sqlfiddle.com/#!2/85ae6
In there you'll see that the column instrument and tenor do have duplicate values.
I'd say you need to investigate more how unique keys work and what primary keys are.
My suggestion is to re-think your requirements and investigate what needs to be unique and why and have a different structure to support your decision. Composite primary keys, in this case, is not the way to go.
I have two tables. One is called map_life, and the second one is called scripts. The map_life table has a lot of rows, that are identified by a column called lifeid. The rows at the table scripts are identified by the column objectid. I want to create a query that gets all the rows from the table map_life and also adds the column scriptfrom scripts table if lifeidmatches objectid, and that the objecttype is npc.
I created the following query.
SELECT id
,lifeid
,x_pos
,y_pos
,foothold
,min_click_pos
,max_click_pos
,respawn_time
,flags
,script.script
FROM map_life life
LEFT JOIN scripts script
ON script.objectid = life.lifeid
AND script.script_type = 'npc'
However, that query takes a lot of time. Any way I can tune it? Thanks.
EDIT: I have ran EXPLAIN command, there are the results.
"id","select_type","table","type","possible_keys","key","key_len","ref","rows","Extra"
1,"SIMPLE","life","ALL","","","","",47600,""
1,"SIMPLE","script","ref","PRIMARY","PRIMARY","1","const",1834,"Using where"
EDIT 2: Here are the create statmenets of each table.
map_life
DROP TABLE IF EXISTS `mcdb`.`map_life`;
CREATE TABLE `mcdb`.`map_life` (
`id` bigint(21) unsigned NOT NULL AUTO_INCREMENT,
`mapid` int(11) NOT NULL,
`life_type` enum('npc','mob','reactor') NOT NULL,
`lifeid` int(11) NOT NULL,
`life_name` varchar(50) DEFAULT NULL COMMENT 'For reactors, specifies a handle so scripts may interact with them; for NPC/mob, this field is useless',
`x_pos` smallint(6) NOT NULL DEFAULT '0',
`y_pos` smallint(6) NOT NULL DEFAULT '0',
`foothold` smallint(6) NOT NULL DEFAULT '0',
`min_click_pos` smallint(6) NOT NULL DEFAULT '0',
`max_click_pos` smallint(6) NOT NULL DEFAULT '0',
`respawn_time` int(11) NOT NULL DEFAULT '0',
`flags` set('faces_left') NOT NULL DEFAULT '',
PRIMARY KEY (`id`,`lifeid`) USING BTREE,
KEY `lifetype` (`mapid`,`life_type`)
) ENGINE=InnoDB AUTO_INCREMENT=47557 DEFAULT CHARSET=latin1;
scripts
DROP TABLE IF EXISTS `mcdb`.`scripts`;
CREATE TABLE `mcdb`.`scripts` (
`script_type` enum('npc','reactor','quest','item','map','map_enter','map_first_enter') NOT NULL,
`helper` tinyint(3) NOT NULL DEFAULT '-1' COMMENT 'Represents the quest state for quests, and the index of the script for NPCs (NPCs may have multiple scripts).',
`objectid` int(11) NOT NULL DEFAULT '0',
`script` varchar(40) NOT NULL DEFAULT '',
PRIMARY KEY (`script_type`,`helper`,`objectid`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1 COMMENT='Lists all the scripts that belong to NPCs/reactors/etc. ';
You should probably add an index to the 'script_type' field depending on the type. If it's not using a type that can be indexed, you should change the type if possible and index
Here is a link that discusses more about indexes with MySQL, http://dev.mysql.com/doc/refman/5.0/en/mysql-indexes.html
Your primary key on scripts is:
PRIMARY KEY (`script_type`,`helper`,`objectid`)
The order of multi-column keys is important.
From the docs:
Any index that does not span all AND levels in the WHERE clause is not
used to optimize the query. In other words, to be able to use an
index, a prefix of the index must be used in every AND group.
Your primary key on scripts does include both the script_type and objectid columns, which are both used in the join's ON clause:
ON script.objectid = life.lifeid
AND script.script_type = 'npc'
but the primary key also includes the helper column between those two, so MySQL can only use the primary key index for searching using the first column (script_type).
So, for every join, MySQL must search through all scripts records where script_type is 'npc' to find the particular objectid record to join on.
MySQL could full utilize the primary key index if your ON clause included all three columns like this:
ON script.objectid = life.lifeid
AND script.helper = 1
AND script.script_type = 'npc'
If you often query the scripts table without specifying the helper column, consider changing the order of the columns in the primary key to put the helper column last:
PRIMARY KEY (`script_type`,`objectid`,`helper`)
Then, your original ON clause is appropriate for the index because the index prefix includes all of the columns in your predicate (script_type,objectid):
ON script.objectid = life.lifeid
AND script.script_type = 'npc'
Alternatively, add an additional index with just the two columns mentioned in the ON clause:
KEY `scrypt_type_objectid` (`script_type`,`objectid`)
I need to add multiple records to a mysql database. I tried with multiple queries and its working fine, but not efficient. So I tried it with just one query like below,
INSERT INTO data (block, length, width, rows) VALUES
("BlockA", "200", "10", "20"),
("BlockB", "330", "8", "24"),
("BlockC", "430", "7", "36")
ON DUPLICATE KEY UPDATE
block=VALUES(block),
length=VALUES(length),
width=VALUES(width),
rows=VALUES(rows)
But it always update the table (columns are block_id, block, length, width, rows).
Should I do any changes on the query with adding block_id also. block_id is the primary key. Any help would be appreciated.
I've run your query without any problem, are you sure you don't have other keys defined with the data table ? And also make sure you have 'auto increment' set for the id field. without auto_increment, the query always update existing row
***** Updated **********
Sorry I've mistaken your questions. Yes, with only one auto_increment key, you query will always insert new rows instead of updating existing one ( because the primary key is the only way to detect 'existing' / duplication ), since the key is auto_increment, there's never a duplication if the primary key is not given in the insert query.
I think what you want to achieve is different, you might want to set up composite unique key on all fields (i.e. block, field, width, rows )
By the way, i've set up a SQL fiddle for you.
http://sqlfiddle.com/#!2/e7216/1
The syntax to add the unique key:
CREATE TABLE `data` (
`id` int(11) unsigned NOT NULL AUTO_INCREMENT,
`block` varchar(10) DEFAULT NULL,
`length` int(11) DEFAULT NULL,
`width` int(11) DEFAULT NULL,
`rows` int(11) DEFAULT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `uniqueme` (`block`,`length`,`width`,`rows`)
) ENGINE=InnoDB AUTO_INCREMENT=2 DEFAULT CHARSET=utf8;