MySQL version 5.7, ONLY_FULL_GROUP_BY is enabled
Please tell me an effective way to solve the following problem: it is necessary to make an entry in the table only if there are no matches in the record for several fields. If there is a matching record, then we update it with new data.
The ON DUPLICATE KEY UPDATE statement does not suit me, because the type and uuid_session fields are not unique (for example, there may be several records with different type, but with the same uuid_session).
Here is an example of a fake-query to better understand my question:
INSERT INTO cameraStatus(time, uuid_session)
VALUES ('2022-12-14 16:01:00', '01234567-8901-2345-6789-012345678901')
ON DUPLICATE KEY UPDATE (type, uuid_session) = ("WORK", "55555555-8901-2345-6789-012345678901");
My table:
CREATE TABLE `cameraStatus` (
`id` int NOT NULL,
`camera_id` int NOT NULL DEFAULT '0',
`time` timestamp NOT NULL DEFAULT '2021-12-31 21:00:00',
`type` varchar(50) COLLATE utf8_unicode_ci NOT NULL DEFAULT 'INFO',
`message` mediumtext COLLATE utf8_unicode_ci,
`uuid_session` varchar(36) CHARACTER SET utf8mb3 COLLATE utf8_unicode_ci DEFAULT '01234567-8901-2345-6789-012345678901'
)
Related
I'm renaming a bunch of columns in a lot of my tables and changing their data types for a major system update, and I haven't had many issues except for this one.
Error Code: 1054. Unknown column 'FactoryID' in 'factories'
show create table `factories`;
CREATE TABLE `factories` (
`FactoryID` char(36) COLLATE utf8mb4_unicode_ci NOT NULL,
`ParentFactoryID` char(36) COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`Name` varchar(256) COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`Notes` varchar(1024) COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`DateTimeAdded` datetime DEFAULT NULL,
`CountryID` smallint(5) unsigned NOT NULL,
`ListID` varchar(64) COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`Deleted` int(1) DEFAULT '0',
PRIMARY KEY (`FactoryID`),
UNIQUE KEY `FactoryID` (`FactoryID`),
KEY `ParentFactoryID` (`ParentFactoryID`),
KEY `CountryID` (`CountryID`),
CONSTRAINT `factories[CountryID]` FOREIGN KEY (`CountryID`) REFERENCES `countries` (`CountryID`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
ALTER TABLE `sterling`.`factories`
CHANGE COLUMN `CountryID` `CountryID` SMALLINT(5) UNSIGNED NOT NULL AFTER `FactoryID`,
CHANGE COLUMN `ParentFactoryID` `_OriginalFactoryID` CHAR(36) CHARACTER SET 'utf8mb4' COLLATE 'utf8mb4_unicode_ci' NULL DEFAULT NULL AFTER `ListID`,
CHANGE COLUMN `DateTimeAdded` `__Added` TIMESTAMP(6) NOT NULL DEFAULT CURRENT_TIMESTAMP(6) AFTER `__Active`,
CHANGE COLUMN `Deleted` `__Active` TINYINT(1) NOT NULL DEFAULT 1 ,
ADD COLUMN `__Updated` TIMESTAMP(6) NULL DEFAULT NULL ON UPDATE CURRENT_TIMESTAMP(6) AFTER `__Added`,
drop primary key,
change column`FactoryID` `#FactoryID`char(36)null after`__Updated`,
add column`FactoryID`binary(8)not null first,
add index`#FactoryID`(`#FactoryID`);
Am I missing an order of operations sort of thing? If all of those changes happen in order, I'm not sure what exactly the problem is, since at no time that FactoryID is referenced does it not exist.
The columnname that after refers to is supposed to be the new name of the column in the final table. It doesn't matter where in the statement you change the name.
And it actually even makes this following statement fail:
alter table tablename
add column b int after a,
drop column a
Error Code: 1054. Unknown column 'a' in 'tablename'
as in the final table, there will be no column a anymore, so it is invalid even if you drop a only after you already added column b.
In your case, you would need to change
CHANGE COLUMN `CountryID` `CountryID` ... AFTER `FactoryID`,
to
CHANGE COLUMN `CountryID` `CountryID` ... AFTER `#FactoryID`,
in order to anticipate that you will (later) rename the column FactoryID to #FactoryID.
To make it complete: in after, you cannot refer to a column that you will add later. For example, in the end of your statement, you actually add another column FactoryID, but you cannot yet refer to it here (otherwise, the query would not have failed). You could add that column first though (and even before you rename the original FactoryId, MySQL allows you to swap columnnames that way). CHANGE COLUMN CountryID CountryID ... AFTER FactoryID would then work, but would refer to the new column (so in total, CountryID would be the 2nd column, which may or may not be what you intended).
I don't know if it is officially documented somewhere, you will probably have to take it as convention, but it has "always" been that way.
I have 2 MySQL table name uploads and uploads_log.
uploads table has a field name json_values (datatype: longtext)
uploads_log table has 2 fields old_value, new_value (both datatype: longtext)
On After UPDATE of uploads table I have written a trigger which just put the whole content of uploads.json_values into uploads_log table's old_value, new_value.
trigger is
BEGIN
IF (NEW.json_values != OLD.json_values) THEN
INSERT INTO uploads_log (`file_id`, `user_id`, `field_name`, `old_value`, `new_value`, `ip`, `created_at`)
VALUES (OLD.`file_id`,
OLD.`user_id`,
'json_values',
OLD.json_values,
NEW.json_values,
NEW.user_ip,
NOW());
END IF;
END
My issue is: When I'm editing small string in uploads.json_values my trigger is working fine, but when Im editting some realy long string like 378369 characters long. I'm getting the following error.
SQLSTATE[42000]: Syntax error or access violation: 1118 Row size too large (> 8126). Changing some columns to TEXT or BLOB or using ROW_FORMAT=DYNAMIC or ROW_FORMAT=COMPRESSED may help. In current row format, BLOB prefix of 768 bytes is stored inline.
I try to debug the issue I removed the trigger and EDITED uploads.json_values with long string it workes fine, and I manually INSERTED that long string into uploads_log.old_value then also it works fine, So the issue is with the trigger.
Is trigger has some limitataion of length?
Both the table uses Storage Engine: InnoDB and MySQL Version is 5.6.21.
uploads table Structure
CREATE TABLE `uploads` (
`file_id` int(11) unsigned NOT NULL AUTO_INCREMENT,
`user_id` int(11) unsigned NOT NULL,
`json_values` longtext COLLATE utf8_unicode_ci NOT NULL,
`read_values` longtext COLLATE utf8_unicode_ci,
`user_ip` varchar(50) COLLATE utf8_unicode_ci DEFAULT NULL,
PRIMARY KEY (`file_id`),
KEY `user_id` (`user_id`)
) ENGINE=InnoDB AUTO_INCREMENT=34444 DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci
uploads_log table Structure
CREATE TABLE `uploads_log` (
`action_id` bigint(20) unsigned NOT NULL AUTO_INCREMENT,
`file_id` int(11) unsigned DEFAULT NULL,
`user_id` int(11) unsigned DEFAULT NULL,
`field_name` varchar(255) COLLATE utf8_unicode_ci DEFAULT NULL,
`old_value` longtext COLLATE utf8_unicode_ci,
`new_value` longtext COLLATE utf8_unicode_ci,
`ip` varchar(30) COLLATE utf8_unicode_ci DEFAULT NULL,
`created_at` datetime DEFAULT NULL,
PRIMARY KEY (`action_id`)
) ENGINE=InnoDB AUTO_INCREMENT=6 DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci
I found this question and this one, but it is not related to UPDATE trigger.
Any help/suggestion will be very much appreciated.
Thanks.
I also had some similar issue but not with trigger this question helped me out.
Change innodb_log_file_size value located in C:\xampp\mysql\bin\my.ini (if you are using XAMPP) to something higher than 5M.
or as pointed out by #Vatev you can set innodb_log_file_size = 128M
You can use this MySQL command to get the innodb_log_file_size value, it will give you the result in Byte.
I have a large table with ~3 million records in a MySQL database. I am trying to find duplicate rows in this table using the following query -
SELECT package_id
FROM version
WHERE metadata IS NOT NULL AND metadata <> '{}'
GROUP BY package_id, metadata HAVING COUNT(package_id) > 1
This query takes ~23 seconds to run on the database. Our database host however kill any query taking larger than 3 seconds using pt-kill. So I need to find a way to break this query down, such as each of the subpart would be a separate query and each one takes less than 3 seconds. Adding just a LIMIT constraint doesn't do it for the query, so how do I break a query to work on different parts of the table.
Result of SHOW CREATE TABLE version
CREATE TABLE `version` (
`id` bigint(20) unsigned NOT NULL AUTO_INCREMENT,
`package_id` bigint(20) unsigned NOT NULL,
`version_number` int(11) unsigned NOT NULL,
`current_state_id` tinyint(2) unsigned NOT NULL,
`md5sum` varchar(32) CHARACTER SET utf8 COLLATE utf8_general_cs NOT NULL DEFAULT '',
`uri` varchar(1024) CHARACTER SET utf8 COLLATE utf8_general_cs NOT NULL DEFAULT '',
`filename` varchar(255) CHARACTER SET utf8 COLLATE utf8_general_cs NOT NULL DEFAULT '',
`size` bigint(11) unsigned NOT NULL DEFAULT '0',
`metadata` varchar(1024) CHARACTER SET utf8 COLLATE utf8_general_cs DEFAULT NULL,
`storage_type_id` tinyint(2) unsigned NOT NULL DEFAULT '1',
PRIMARY KEY (`id`),
UNIQUE KEY `idx_version_package_id_version_number` (`package_id`,`version_number`),
KEY `idx_version_md5sum` (`md5sum`),
KEY `idx_version_metadata` (`metadata`(255)),
KEY `idx_version_current_state_id` (`current_state_id`),
KEY `storage_type_id` (`storage_type_id`),
CONSTRAINT `_fk_version_current_state_id` FOREIGN KEY (`current_state_id`) REFERENCES `state` (`id`),
CONSTRAINT `_fk_version_package_id` FOREIGN KEY (`package_id`) REFERENCES `package` (`id`) ON DELETE CASCADE
) ENGINE=InnoDB AUTO_INCREMENT=3248761 DEFAULT CHARSET=utf8
As can be seen there are many indexes on the table including index on Package_id + Version_number combination of field. The problem is that this table is only going to get bigger and I don't think Optimization even if it pulls me back in 3 second range would scale. So I need a way where I can partition this table and run on queries on separate parts.
Steps to improve speed.
Create Table version_small with just columns id and package_id with index on package_id.
insert into version_small select id and package_id from version;
Run your original query on optimised table above - should be much faster on smaller table.
OR
Create Table version_small with just columns id and package_id, and a int counter with unique index on package_id.
insert into version_small select id and package_id from version, on duplicate key increment counter;
The rows with counter>1 are package_id that have more than one entry.
I just did an export of a MySQL database in order to duplicate it on another server. Looking at the sql script I see the following:
CREATE TABLE `X` (
`id` int(11) unsigned NOT NULL AUTO_INCREMENT,
`name` varchar(50) COLLATE utf8_unicode_ci DEFAULT NULL,
`email` varchar(50) COLLATE utf8_unicode_ci DEFAULT NULL
PRIMARY KEY (`id`),
KEY `name_index` (`name`)
) ENGINE=InnoDB AUTO_INCREMENT=8366
DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci;
Do I need to use this per column when collate is set for the table?
I think the reason for why each column has this, was because I wanted to make sure I could store locale values such as æ, ø å.
Since your column-level collations are all the same, setting the collation at the table level will have the same effect.
Note that collation has no effect on what you can store; rather, it affects how the results are sorted. The only reason you would want a different collation per column would be if your table contains columns with data for different locales requiring a different sort order.
I'm trying to insert the distinct values of one field of the table requests into the only field of the table pastmonth
CREATE TABLE `pastmonth` (
`video_id` char(5) DEFAULT NULL,
PRIMARY KEY (`video_id`)
);
INSERT INTO pastmonth (video_id)
SELECT DISTINCT (video_id)
FROM requests
WHERE exec_datetime >= NOW() - 60*24*60*60;
However I get this error
Error Code: 1062. Duplicate entry 'abcde' for key 'PRIMARY'
I don't believe there are duplicate entries for the value abcde in the table requests because the following query indicates there is only 1 entry with that but it should not matter if there was because I'm SELECTING the DISTINCT values anyways.
SELECT COUNT(*) FROM requests WHERE video_id = 'abcde';
COUNT(*)
1
I have a non-unique index on requests.video_id but I think that is irrelevant to the problem I have. Is it possible that I have a corrupt btree index?
EDIT
CREATE TABLE `requests` (
`request_id` bigint(20) NOT NULL AUTO_INCREMENT,
`video_id` char(5) CHARACTER SET latin1 COLLATE latin1_bin DEFAULT NULL,
`exec_datetime` datetime DEFAULT NULL,
PRIMARY KEY (`request_id`),
KEY `exec_datetime` (`exec_datetime`),
) ENGINE=InnoDB AUTO_INCREMENT=1 DEFAULT CHARSET=latin1
In the requests table, the video_id field is defined as:
`video_id` char(5) CHARACTER SET latin1 COLLATE latin1_bin DEFAULT NULL,
But in the new table, it's defined as:
`video_id` char(5)
There might be a difference in character set and corresponding collation. Make sure both fields have exactly the same character set and collation.