I have dug through SO questions and none address my specific issue ... I have read the following relevant threads: Here, to no avail.
I have a simple table with the following data:
|-----------------------------------------------------------------------------|
| id | contractor_id | section_id | page_id | modified | timestamp |
|-----------------------------------------------------------------------------|
Here is the create statement:
CREATE TABLE `reusable_page_modified` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`contractor_id` int(11) DEFAULT NULL,
`section_id` int(11) DEFAULT NULL,
`page_id` int(11) DEFAULT NULL,
`modified` int(1) NOT NULL DEFAULT '1',
`timestamp` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
PRIMARY KEY (`id`),
UNIQUE KEY `id_UNIQUE` (`id`)
) ENGINE=InnoDB AUTO_INCREMENT=13844 DEFAULT CHARSET=utf8;
Now I have 4 rows in the db right now, and they are (I'll leave off id and timestamp since they are auto generated):
|---------------------------------------------------------|
| contractor_id | section_id | page_id | modified |
###########################################################
| 1016 | 309 | 10303 | 0 |
|----------------------------------------------------------
| 1017 | 309 | 10303 | 1 |
|----------------------------------------------------------
| 1073 | 309 | 10303 | 1 |
|----------------------------------------------------------
| 240 | 309 | 10303 | 1 |
|----------------------------------------------------------
I am focusing on the first line where modified is set to 0. What I want to do is set it to 1 if contractor_id, section_id and page_id all exist. If not, enter a new row.
This is what I have:
INSERT INTO `reusable_page_modified`
(contractor_id, section_id, page_id, modified)
VALUES ('1016', '309', '10303', '1')
ON DUPLICATE KEY UPDATE
`section_id` = '309'
AND `page_id` = '10303'
AND `contractor_id` = '1016';
This creates a new row. I think I am not understanding the ON DUPLICATE KEY UPDATE statment the way it was intended. I have read the MySQL documentation Here and still no help. What am I not seeing here?
You are not far from being correct, but your syntax is a bit off. First, if you want ON DUPLICATE KEY to work when a record with duplicate values for the contractor_id, section_id, and page_id columns is inserted, you will need a unique constraint on these three columns. You can add one using the following:
ALTER TABLE reusable_page_modified
ADD CONSTRAINT u_cnst UNIQUE (contractor_id, section_id, page_id);
Next, for the actual INSERT statement you would use what you have is close, except that you update all the columns which don't require any updating. Instead, leave the three columns alone and instead update the modified column to 1:
INSERT INTO reusable_page_modified
(contractor_id, section_id, page_id, modified)
VALUES ('1016', '309', '10303', '1')
ON DUPLICATE KEY UPDATE
modified = 1
Related
We have the following table and we ran into an issue that our unique key is not working (as expected).
CREATE TABLE `documentation_photos` (
`commissionID` smallint(5) unsigned NOT NULL,
`periodID` smallint(5) unsigned NOT NULL,
`carreirID` int(11) NOT NULL,
`groupID` smallint(5) unsigned DEFAULT NULL,
`order` tinyint(4) NOT NULL,
`file` varchar(200) NOT NULL,
UNIQUE KEY `idZak_UNIQUE` (`commissionID`,`periodID`,`carreirID`,`groupID`,`order`),
KEY `ncis & obdobi & idZak` (`commissionID`,`periodID,`carreirID``),
KEY `fotodoc_skupina` (`idSkup`),
CONSTRAINT `fotodoc_` FOREIGN KEY (`groupID`) REFERENCES `groups` (`groupID`),
CONSTRAINT `fotodoc_zaknos` FOREIGN KEY (`commissionID`, `periodID`, `carreirID`) REFERENCES `carrier_of_commission` (`commissionID`, `periodID`, `carreirID`) ON DELETE CASCADE
);
I saw the issue, that describes that NULL is considered unique every time it occurs, but it causes a lot of problems for us. The only one I found was for Postgres...
|--------------|----------|-----------|---------|-------|---------|
| commissionID | periodID | carrierID | groupID | order | file |
|--------------|----------|-----------|---------|-------|---------|
| 1140 | 117 | 2235 | null | 1 | photo-1 |
|--------------|----------|-----------|---------|-------|---------|
| 1140 | 117 | 2235 | null | 1 | photo-1 | -- This is duplicate
|--------------|----------|-----------|---------|-------|---------|
| 1140 | 117 | 2235 | null | 2 | photo-2 |
|--------------|----------|-----------|---------|-------|---------|
| 1140 | 117 | 2235 | null | 1 | photo-1 | -- This is duplicate
|--------------|----------|-----------|---------|-------|---------|
I want to treat null as a single value.
We would like to have working unique/primary key and keep the foreign key if possible.
Is there some known solution for this in MySQL?
Each NULL is unique (two NULLs are not equal) - the row which contains NULL in index expression is not checked for uniqueness, the row which contains NULL in FK expressions is not checked for reference integrity.
Is there some known solution for this in MySQL?
Use generated column which replaces NULL with some definite but logically impossible literal value, use this column in index/FK expression instead of nullale column.
Following is my table schema, we are using mysql 5.7 version
CREATE TABLE rule_reports (
pkey int(11) NOT NULL AUTO_INCREMENT,
email varchar(250) DEFAULT NULL,
domain varchar(250) NOT NULL,
rule_id bigint(20) unsigned NOT NULL,
rule_type varchar(250) NOT NULL,
log_time datetime DEFAULT NULL,
count int(11) DEFAULT NULL,
PRIMARY KEY (pkey),
KEY dwl (domain,rule_id,log_time)
)
I want to increment count column instead of new row in the table, if
combination of values of domain,rule_id,rule_type already exists in the table row
Sample rows of table
+------+-------------------------+---------+------------------+------------------+-------------------------+----------
| pkey | email | domain | rule_id | rule_type | log_time | count
+------+-------------------------+---------+------------------+------------------+-------------------------+-----------
| 1 | user1#yopmail.com | user1 | 566 | type1 | 2016-09-13 17:23:02.000 | 1
| 2 | user2#yopmail.com | user2 | 567 | type2 | 2016-09-13 17:23:02.000 | 1
-----------------------------------------------------------------------------------------------------------------------
Suppose if statement like below should not create a row because same values for domain,rule_id,rule_type already exists in table so I need a count column increment here
insert into rule_reports(domain,rule_id,rule_type,count) values('user1',566,'type1',1)
Statement like below should create a new row in table
insert into rule_reports(domain,rule_id,rule_type,count) values('user3',568,'type3',1)
Add a unique index on the columns involved, and the use ON DUPLICATE KEY UPDATE to increment the count if the record already exists.
ALTER TABLE rule_reports ADD UNIQUE unique_index(domain, rule_id, rule_type);
INSERT INTO rule_reports (domain, rule_id, rule_type, count)
VALUES ('user1', 566, 'type1', 1)
ON DUPLICATE KEY UPDATE count = count + 1
so I have two tables linked by the key 'skillid':
skills
+-----------+-------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+-----------+-------------+------+-----+---------+----------------+
| skillid | int(11) | NO | PRI | NULL | auto_increment |
| skillname | varchar(30) | NO | | NULL | |
+-----------+-------------+------+-----+---------+----------------+
students_skills
+-----------+---------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+-----------+---------+------+-----+---------+----------------+
| ssid | int(11) | NO | PRI | NULL | auto_increment |
| studentid | int(11) | NO | MUL | NULL | |
| skillid | int(11) | NO | MUL | NULL | |
+-----------+---------+------+-----+---------+----------------+
I'm trying to insert multiple rows into the table skills, and then insert these into student_skills based on the ID that was created. I've been looking into using the LAST_INSERT_ID() function:
INSERT INTO skills (skillid , skillname)
VALUES(NULL,'being grateful for help'); # generate ID by inserting NULL
INSERT INTO students_skills (ssid, studentid, skillid)
VALUES(LAST_INSERT_ID(),'1', '2'); # use ID in second table
But I couldn't figure out how to do this for multiple rows at once in one mysql table. I get an error when i simply duplicate the above 4 lines for every row.
ERROR: #1452 - Cannot add or update a child row: a foreign key
constraint fails (empology.students_skills, CONSTRAINT
students_skills_ibfk_2 FOREIGN KEY (skillid) REFERENCES skills
(skillid))
Am I on the right lines or not? I looked into joins also but this method made more sense to me.
Thanks for any help or useful links.
You have to make sure to use multiple-row insert syntax so that the LAST_INSERT_ID() stays consistent even though you're auto-incrementing another column:
INSERT INTO skills VALUES (NULL, 'test');
Say the skillid generated was 1, you can then do:
INSERT INTO student_skills VALUES
(NULL, 1, LAST_INSERT_ID()),
(NULL, 2, LAST_INSERT_ID()),
(NULL, 3, LAST_INSERT_ID()),
(NULL, 4, LAST_INSERT_ID());
The value returned by LAST_INSERT_ID() will consistently stay the same (1) throughout all four rows.
However, if you execute multiple inserts as standalone statements, LAST_INSERT_ID() will change as it will instead contain the generated auto-incremented value of each insert:
INSERT INTO student_skills VALUES (NULL, 1, LAST_INSERT_ID());
INSERT INTO student_skills VALUES (NULL, 2, LAST_INSERT_ID());
INSERT INTO student_skills VALUES (NULL, 3, LAST_INSERT_ID());
INSERT INTO student_skills VALUES (NULL, 4, LAST_INSERT_ID());
Where LAST_INSERT_ID() is the generated id of the immediate previous insert.
Take a look at this SQLFiddle Demo
Since students_skills.ssid is an AUTO_INCREMENT column, your second insert looks wrong. It seems you want the following:
INSERT INTO skills (skillid , skillname)
VALUES(NULL,'being grateful for help'); # generate ID by inserting NULL
INSERT INTO students_skills (ssid, studentid, skillid)
VALUES(NULL,'1', LAST_INSERT_ID()); # use ID in second table
It would be helpful to see the output of
SHOW CREATE TABLE skills;
SHOW CREATE TABLE students_skills;
to see the FOREIGN KEYs.
UPDATE TO SHOW OUTPUTS
+--------+------------------------------------------------------------------------------
| Table | Create Table
+--------+------------------------------------------------------------------------------
| skills | CREATE TABLE `skills` (
`skillid` int(11) NOT NULL AUTO_INCREMENT,
`skillname` varchar(30) NOT NULL,
PRIMARY KEY (`skillid`)
) ENGINE=InnoDB AUTO_INCREMENT=8 DEFAULT CHARSET=latin1 |
+--------+------------------------------------------------------------------------------
+-----------------+---------------------------------------------------------------------
| Table | Create Table
+-----------------+---------------------------------------------------------------------
| students_skills | CREATE TABLE `students_skills` (
`ssid` int(11) NOT NULL AUTO_INCREMENT,
`studentid` int(11) NOT NULL,
`skillid` int(11) NOT NULL,
PRIMARY KEY (`ssid`),
KEY `studentid` (`studentid`),
KEY `skillid` (`skillid`),
CONSTRAINT `students_skills_ibfk_1` FOREIGN KEY (`studentid`) REFERENCES `students` (`studentid`),
CONSTRAINT `students_skills_ibfk_2` FOREIGN KEY (`skillid`) REFERENCES `skills` (`skillid`)
) ENGINE=InnoDB AUTO_INCREMENT=7 DEFAULT CHARSET=latin1 |
+-----------------+--------------------------------------------------------------------
Example table content
'main'
| id | total |
| 1 | 10 |
| 2 | 20 |
'timed'
| id | id_main | date_from | date_to | total |
| 1 | 2 | 2012-03-29 | 2012-04-29 | 50 |
Desired result
| id | total |
| 1 | 10 |
| 2 | 50 |
Not exactly working query
SELECT main.id AS id, COALESCE(timed.total, main.total) AS total
FROM main
LEFT JOIN timed
ON main.id = timed.id_main
WHERE SYSDATE() BETWEEN timed.date_from AND timed.date_to
Result
| id | total |
| 2 | 50 |
In tables 'main' and 'timed' 'total' field will never be NULL.
In some 'timed' records there will be no relative 'id_main', or there will be few, but they will differ, 'date_from' 'date_to' never intersect.
Table 'main' is large, but in 'timed' will always be two or three relative records.
CREATE TABLE `main` (
`id` int(11) unsigned NOT NULL AUTO_INCREMENT,
`total` decimal(10,2) unsigned NOT NULL DEFAULT '0.00',
PRIMARY KEY (`id`)
) ENGINE=InnoDB;
INSERT INTO `main` VALUES (1,10);
INSERT INTO `main` VALUES (2,20);
CREATE TABLE `timed` (
`id` int(11) unsigned NOT NULL AUTO_INCREMENT,
`id_main` int(11) unsigned NOT NULL DEFAULT '0',
`date_from` date DEFAULT NULL,
`date_to` date DEFAULT NULL,
`total` decimal(10,2) unsigned NOT NULL DEFAULT '0.00',
PRIMARY KEY (`id`),
KEY `link` (`id_main`)
) ENGINE=InnoDB;
INSERT INTO `timed` VALUES (1,2,'2012-03-29','2012-03-30',50);
ALTER TABLE `timed`
ADD CONSTRAINT `link` FOREIGN KEY (`id_main`)
REFERENCES `main` (`id`) ON DELETE CASCADE ON UPDATE CASCADE;
Sorry for my english.
You should move the date condition in the join condition:
SELECT main.id AS id, COALESCE(timed.total, main.total) AS total
FROM main
LEFT JOIN timed
ON main.id = timed.id_main and SYSDATE() BETWEEN timed.date_from AND timed.date_to
In your query, those rows not matched are filtered out by the WHERE condition because timed.date_form and timed.date_to are null, so sysdate can't be between them :)
I have this query (shown below) which currently uses temporary and filesort in order to generate a grouped by set of ordered results. I would like to get rid of their usage if possible. I have looked into the underlying indexes used in this query and I just can't see what is missing.
SELECT
b.institutionid AS b__institutionid,
b.name AS b__name,
COUNT(DISTINCT f2.facebook_id) AS f2__0
FROM education_institutions b
LEFT JOIN facebook_education_matches f ON b.institutionid = f.institutionid
LEFT JOIN facebook_education f2 ON f.school_uid = f2.school_uid
WHERE
(
b.approved = '1'
AND f2.facebook_id IN ( [lots of facebook ids here ])
)
GROUP BY b__institutionid
ORDER BY f2__0 DESC
LIMIT 10
Here is the output for EXPLAIN EXTENDED :
+----+-------------+-------+--------+--------------------------------+----------------+---------+----------------------------------+------+----------+----------------------------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | filtered | Extra |
+----+-------------+-------+--------+--------------------------------+----------------+---------+----------------------------------+------+----------+----------------------------------------------+
| 1 | SIMPLE | f | index | PRIMARY,institutionId | institutionId | 4 | NULL | 308 | 100.00 | Using index; Using temporary; Using filesort |
| 1 | SIMPLE | f2 | ref | facebook_id_idx,school_uid_idx | school_uid_idx | 9 | f.school_uid | 1 | 100.00 | Using where |
| 1 | SIMPLE | b | eq_ref | PRIMARY | PRIMARY | 4 | f.institutionId | 1 | 100.00 | Using where |
+----+-------------+-------+--------+--------------------------------+----------------+---------+----------------------------------+------+----------+----------------------------------------------+
The CREATE TABLE statements for each table are shown below so you know the schema.
CREATE TABLE facebook_education (
education_id int(11) NOT NULL AUTO_INCREMENT,
name varchar(255) DEFAULT NULL,
school_uid bigint(20) DEFAULT NULL,
school_type varchar(255) DEFAULT NULL,
year smallint(6) DEFAULT NULL,
facebook_id bigint(20) DEFAULT NULL,
degree varchar(255) DEFAULT NULL,
PRIMARY KEY (education_id),
KEY facebook_id_idx (facebook_id),
KEY school_uid_idx (school_uid),
CONSTRAINT facebook_education_facebook_id_facebook_user_facebook_id FOREIGN KEY (facebook_id) REFERENCES facebook_user (facebook_id)
) ENGINE=InnoDB AUTO_INCREMENT=484 DEFAULT CHARSET=utf8;
CREATE TABLE facebook_education_matches (
school_uid bigint(20) NOT NULL,
institutionId int(11) NOT NULL,
created_at timestamp NULL DEFAULT NULL,
updated_at timestamp NULL DEFAULT NULL ON UPDATE CURRENT_TIMESTAMP,
PRIMARY KEY (school_uid),
KEY institutionId (institutionId),
CONSTRAINT fk_facebook_education FOREIGN KEY (school_uid) REFERENCES facebook_education (school_uid) ON DELETE CASCADE ON UPDATE CASCADE,
CONSTRAINT fk_education_institutions FOREIGN KEY (institutionId) REFERENCES education_institutions (institutionId) ON DELETE CASCADE ON UPDATE CASCADE
) ENGINE=InnoDB DEFAULT;
CREATE TABLE education_institutions (
institutionId int(11) NOT NULL AUTO_INCREMENT,
name varchar(100) NOT NULL,
type enum('School','Degree') DEFAULT NULL,
approved tinyint(1) NOT NULL DEFAULT '0',
deleted tinyint(1) NOT NULL DEFAULT '0',
normalisedName varchar(100) NOT NULL,
created_at timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
PRIMARY KEY (institutionId)
) ENGINE=InnoDB AUTO_INCREMENT=101327 DEFAULT CHARSET=utf8;
Any guidance would be greatly appreciated.
The filesort probably happens because you have no suitable index for the ORDER BY
It's mentioned in the MySQL "ORDER BY Optimization" docs.
What you can do is load a temp table, select from that afterwards. When you load the temp table, use ORDER BY NULL. When you select from the temp table, use ORDER BY .. LIMIT
The issue is that group by adds an implicit order by <group by clause> ASC unless you disable that behavior by adding a order by null.
It's one of those MySQL specific gotcha's.
I can see two possible optimizations,
b.approved = '1' - You definitely need an index on approved column for quick filtering.
f2.facebook_id IN ( [lots of facebook ids here ]) ) - Store the facebook ids in a temp table,. Then create an index on the temp table and then join with the temp table instead of using IN clause.