I don't know a lot of mysql and have an error in my sql script. Currently running mysql 8.0.24. Does anyone know what might be the problem?
Error:
https://prnt.sc/226xk5x
Sql:
-- Dumping structure for table gtav_rp2._vehicle
CREATE TABLE IF NOT EXISTS `_vehicle` (
`id` int(11) unsigned NOT NULL AUTO_INCREMENT,
`cid` int(11) unsigned NOT NULL,
`vin` varchar(50) NOT NULL DEFAULT '',
`type` varchar(50) NOT NULL DEFAULT '',
`size` int(11) NOT NULL,
`plate` varchar(50) NOT NULL DEFAULT '',
`model` varchar(50) NOT NULL DEFAULT '',
`name` varchar(50) DEFAULT NULL,
`garage` varchar(59) DEFAULT NULL,
`state` varchar(50) DEFAULT NULL,
`appearance` longtext CHARACTER SET utf8mb4 COLLATE utf8mb4_bin DEFAULT NULL CHECK (json_valid('appearance')),
`mods` longtext CHARACTER SET utf8mb4 COLLATE utf8mb4_bin DEFAULT NULL CHECK (json_valid('mods')),
`data` longtext DEFAULT NULL,
`damage` longtext CHARACTER SET utf8mb4 COLLATE utf8mb4_bin DEFAULT NULL CHECK (json_valid('damage')),
`degredation` longtext CHARACTER SET utf8mb4 COLLATE utf8mb4_bin DEFAULT NULL CHECK (json_valid('degredation')),
PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;
-- Dumping data for table gtav_rp2._vehicle: ~0 rows (approximately)
/*!40000 ALTER TABLE `_vehicle` DISABLE KEYS */;
/*!40000 ALTER TABLE `_vehicle` ENABLE KEYS */;
/*!40101 SET SQL_MODE=IFNULL(#OLD_SQL_MODE, '') */;
/*!40014 SET FOREIGN_KEY_CHECKS=IF(#OLD_FOREIGN_KEY_CHECKS IS NULL, 1, #OLD_FOREIGN_KEY_CHECKS) */;
/*!40101 SET CHARACTER
_SET_CLIENT=#OLD_CHARACTER_SET_CLIENT */;
Your CHECK constraint does not reference the column. Or any column, actually. By using single-quotes, you're using a string literal, not a column name.
`appearance` longtext CHARACTER SET utf8mb4 COLLATE utf8mb4_bin DEFAULT NULL
CHECK (json_valid('appearance')),
I assume this is meant to be:
`appearance` longtext CHARACTER SET utf8mb4 COLLATE utf8mb4_bin DEFAULT NULL
CHECK (json_valid(`appearance`)),
Use the right type of quotes for SQL identifiers, not string literals.
In addition, this CHECK constraint would be unnecessary if you used the JSON data type instead of LONGTEXT. The JSON data type already enforces that the content of the column must be valid JSON format.
MySQL's JSON data type already uses utf8mb4 character set and utf8mb4_bin collation.
So your column definition could be simply as follows:
`appearance` JSON
Related
My business scenario is shown in the figure above. A user can create multiple products, a product can have multiple modules, and a module can have multiple parameters. The parameters include variable types, variable names, and variable values.
Before starting I thought the query speed of mptt was better than 2D relational table, but the result is completely opposite.
I now have two data table designs.
Option Oneļ¼
CREATE TABLE `products` (
`product_id` bigint(20) NOT NULL AUTO_INCREMENT,
`product_name` varchar(30) CHARACTER SET utf8 COLLATE utf8_general_ci NULL DEFAULT NULL,
`cfdversion` bigint(20) NULL DEFAULT NULL,
`product_info` varchar(30) CHARACTER SET utf8 COLLATE utf8_general_ci NULL DEFAULT NULL,
`is_activated` tinyint(1) NULL DEFAULT NULL,
PRIMARY KEY (`product_id`) USING BTREE
) ENGINE = InnoDB AUTO_INCREMENT = 226 CHARACTER SET = utf8 COLLATE = utf8_general_ci ROW_FORMAT = Dynamic;
CREATE TABLE `person_param` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`product_id` bigint(20) NULL DEFAULT NULL,
`param_name` varchar(20) CHARACTER SET utf8 COLLATE utf8_general_ci NULL DEFAULT NULL,
`var_type` varchar(10) CHARACTER SET utf8 COLLATE utf8_general_ci NULL DEFAULT NULL,
`var_value` varchar(30) CHARACTER SET utf8 COLLATE utf8_general_ci NULL DEFAULT NULL,
`var_name` varchar(30) CHARACTER SET utf8 COLLATE utf8_general_ci NULL DEFAULT NULL,
`is_activated` tinyint(1) NULL DEFAULT 1,
`compute_value` varchar(20) CHARACTER SET utf8 COLLATE utf8_general_ci NULL DEFAULT NULL,
`module_name` varchar(10) CHARACTER SET utf8 COLLATE utf8_general_ci NULL DEFAULT NULL,
PRIMARY KEY (`id`) USING BTREE,
INDEX `product_id`(`product_id`) USING BTREE,
CONSTRAINT `person_param_ibfk_1` FOREIGN KEY (`product_id`) REFERENCES `products` (`product_id`) ON DELETE RESTRICT ON UPDATE RESTRICT
) ENGINE = InnoDB AUTO_INCREMENT = 19 CHARACTER SET = utf8 COLLATE = utf8_general_ci ROW_FORMAT = Dynamic;
I connect the id of the product information with the parameter table.
Option two:
products table is same just table name is different.
person_paramlike this:
CREATE TABLE `mptt_param` (
`node_id` bigint(20) NOT NULL AUTO_INCREMENT,
`node_name` varchar(30) CHARACTER SET utf8 COLLATE utf8_general_ci NULL DEFAULT NULL,
`lft` bigint(20) NOT NULL,
`rgt` bigint(20) NOT NULL,
`node_level` bigint(20) NOT NULL,
PRIMARY KEY (`node_id`) USING BTREE
) ENGINE = InnoDB AUTO_INCREMENT = 1 CHARACTER SET = utf8 COLLATE = utf8_general_ci ROW_FORMAT = Dynamic;
I added 200 products with 3 modules per product and 10 parameters per module.
option 1's sql statement:SELECT * from `products` a RIGHT JOIN `person_param` b ON a.product_id=b.product_id WHERE a.product_id=246;
option 2's sql statement:SELECT * FROM mptt_param WHERE lft>=(SELECT lft FROM mptt_param WHERE node_name='246') AND rgt<=(SELECT rgt FROM mptt_param WHERE node_name='246') ;
I don't know what the problem is, hope you can give me some advice
I have the following table:
CREATE TABLE `RU_sg1lib_MASTER` (
`ID` int(15) unsigned NOT NULL AUTO_INCREMENT,
`Title` varchar(2000) COLLATE utf8mb4_unicode_ci DEFAULT '',
`VolumeInfo` varchar(100) CHARACTER SET utf8 DEFAULT '',
`Series` varchar(300) CHARACTER SET utf8 DEFAULT '',
`Periodical` varchar(200) CHARACTER SET utf8 DEFAULT '',
`Author` varchar(1000) CHARACTER SET utf8 DEFAULT '',
`Year` varchar(14) CHARACTER SET utf8 DEFAULT '',
`Edition` varchar(60) CHARACTER SET utf8 DEFAULT '',
`Publisher` varchar(400) CHARACTER SET utf8 DEFAULT '',
`City` varchar(100) CHARACTER SET utf8 DEFAULT '',
`Pages` varchar(100) CHARACTER SET utf8 DEFAULT '',
`PagesInFile` int(10) unsigned NOT NULL DEFAULT '0',
`Language` varchar(150) CHARACTER SET utf8 DEFAULT '',
`Topic` varchar(500) CHARACTER SET utf8 DEFAULT '',
`Library` varchar(50) CHARACTER SET utf8 DEFAULT '',
`Issue` varchar(100) CHARACTER SET utf8 DEFAULT '',
`Identifier` varchar(300) CHARACTER SET utf8 DEFAULT '',
`ISSN` varchar(9) CHARACTER SET utf8 DEFAULT '',
`ASIN` varchar(200) CHARACTER SET utf8 DEFAULT '',
`UDC` varchar(200) CHARACTER SET utf8 DEFAULT '',
`LBC` varchar(200) CHARACTER SET utf8 DEFAULT '',
`DDC` varchar(45) CHARACTER SET utf8 DEFAULT '',
`LCC` varchar(45) CHARACTER SET utf8 DEFAULT '',
`Doi` varchar(45) CHARACTER SET utf8 DEFAULT '',
`Googlebookid` varchar(45) CHARACTER SET utf8 DEFAULT '',
`OpenLibraryID` varchar(200) CHARACTER SET utf8 DEFAULT '',
`Commentary` varchar(10000) CHARACTER SET utf8 DEFAULT '',
`DPI` int(6) unsigned DEFAULT '0',
`Color` varchar(1) CHARACTER SET utf8 DEFAULT '',
`Cleaned` varchar(1) CHARACTER SET utf8 DEFAULT '',
`Orientation` varchar(1) CHARACTER SET utf8 DEFAULT '',
`Paginated` varchar(1) CHARACTER SET utf8 DEFAULT '',
`Scanned` varchar(1) CHARACTER SET utf8 DEFAULT '',
`Bookmarked` varchar(1) CHARACTER SET utf8 DEFAULT '',
`Searchable` varchar(1) CHARACTER SET utf8 DEFAULT '',
`Filesize` bigint(20) unsigned NOT NULL DEFAULT '0',
`Extension` varchar(50) CHARACTER SET utf8 DEFAULT '',
`MD5` char(32) CHARACTER SET utf8 DEFAULT '',
`Generic` char(32) CHARACTER SET utf8 DEFAULT '',
`Visible` char(3) CHARACTER SET utf8 DEFAULT '',
`Locator` varchar(733) CHARACTER SET utf8 DEFAULT '',
`Local` int(10) unsigned DEFAULT '0',
`TimeAdded` timestamp NOT NULL DEFAULT '2000-01-01 13:00:00',
`TimeLastModified` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
`Coverurl` varchar(200) CHARACTER SET utf8 DEFAULT '',
`Tags` varchar(500) CHARACTER SET utf8 DEFAULT '',
`IdentifierWODash` varchar(300) CHARACTER SET utf8 DEFAULT '',
`Zstatus_retrieve_TOC` varchar(255) CHARACTER SET utf8 DEFAULT NULL,
`Zstatus_retrieve_classifybyoclc` varchar(255) CHARACTER SET utf8 DEFAULT NULL,
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci
I know from a manual inspection of the data that the Title column contains many duplicates. So, I embarked on removing the duplicates. I did this by:
Step 1: Copy the table with data to another table (temp) using PHPMyadmin
Step 2: TRUNCATE TABLE RU_sg1lib_MASTER
Step 3: There was a Primary Key index on the ID column. So I removed it, and created a Primary Key (length 256) on the Title. The resultant indexes are:
PRIMARY KEY (Title(256)),
KEY classifybyoclc_respcode (classifybyoclc_respcode(256)),
KEY classifybyoclc_calln_lcc (classifybyoclc_calln_lcc),
Step 4: INSERT IGNORE INTO RU_sg1lib_MASTER SELECT * FROM temp
At that time, I did not have a complete knowledge about indexes (especially indexes with specified length). Lately, it occurred to me that because the Title index has specified length of 256, it might cause some values which are not entirely the same but which have the same initial characters to be mistaken as exact duplicates and hence removed.
Am I correct about this?
If yes, how can I remove duplicates accurately for this table which consists of 3+ million records?
Yes, you are correct: If you specify PRIMARY KEY (Title(256)) then the Title column must be unique in the first 256 characters and you would be eliminating rows whose Title column that were identical in those first 256 characters but differed in subsequent character positions. By the way, there was no need to change your primary key. You could have attempted to keep column ID as the primary key and instead define a UNIQUE INDEX for column title without specifying an explicit length. But that would have failed because of a restriction on the largest index size you can have, which for MySQL is 767 bytes (not characters) by default even for MySQL 8.
The maximum index length is 767 bytes (or 3072 for InnoDB tables that use DYNAMIC or COMPRESSED row format). Column Title specifies a collation of utf8mb4_unicode_ci, which means that a character can require up to 4 bytes to store and therefore the largest index you can have is is floor(767/4) or 191 characters (768 characters for InnoDB tables that use DYNAMIC or COMPRESSED row format). So if you have any Title column values whose length is greater than 191 (or 768), you cannot specify an index on this column for the full size. Otherwise, you could just redefine the column to be length 191 (or 768) and your problem is solved.
So assuming that you do have rows where the length of the Title column is greater than the maximum index size you can have, you need an alternate method to guarantee uniqueness since specifying a UNIQUE KEY is no longer an option. You could issue the DELETE statement as suggested by Akina.
But then the problem remains of ensuring that in the future no duplicate Title columns exists. This would require you to first query the table to see if there already exists a row with the given Title value when doing an insert or an update that modifies a Title column value. This could also be done automatically by defining AFTER INSERT and AFTER UPDATE triggers. But since the Title column is unindexed, every row has to be read to determine whether there is uniqueness or not.
I am trying to save Hindi content in db so for that i made changes in table as well with this query
ALTER TABLE group_distribution CHARACTER SET UTF8;
and when i run this query
SHOW VARIABLES LIKE 'character_set%';
I got below result
character_set_client utf8
character_set_connection utf8
character_set_database latin1
character_set_filesystem binary
character_set_results utf8
character_set_server latin1
character_set_system utf8
character_sets_dir /usr/share/mysql/charsets/
What changes i have to made so my DB support other languages too ?
I got the issue ..Issue is related to table definition see below
CREATE TABLE `group_distribution` (
`gd_id` int(11) NOT NULL,
`gd_tweet` varchar(500) CHARACTER SET latin1 NOT NULL,
`gd_ht` varchar(45) CHARACTER SET latin1 DEFAULT NULL,
`gt_created_by` int(11) DEFAULT NULL,
`gt_team_lead` int(11) DEFAULT NULL,
`gt_send_to` int(11) DEFAULT NULL,
`gt_added_dt` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
`gt_update_dt` timestamp NULL DEFAULT NULL,
`gt_active_flag` tinyint(1) NOT NULL,
PRIMARY KEY (`gd_id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
Here you can check it clearly show gd_tweet` varchar(500) CHARACTER SET latin1 NOT NULL, have diffrent characterset thats why created issue now i changed it to
CREATE TABLE `group_distribution` (
`gd_id` int(11) NOT NULL,
`gd_tweet` varchar(500) CHARACTER SET utf8 NOT NULL,
`gd_ht` varchar(45) CHARACTER SET utf8 DEFAULT NULL,
`gt_created_by` int(11) DEFAULT NULL,
`gt_team_lead` int(11) DEFAULT NULL,
`gt_send_to` int(11) DEFAULT NULL,
`gt_added_dt` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
`gt_update_dt` timestamp NULL DEFAULT NULL,
`gt_active_flag` tinyint(1) NOT NULL,
PRIMARY KEY (`gd_id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
ALTER TABLE tbl_name CONVERT TO CHARACTER SET utf8;
I already tried This solution which says
ALTER TABLE title
CHARACTER SET utf8
COLLATE utf8_unicode_ci;
Ok here are some screen shots which might help you.
Update
here's what happens when i insert Japanese characters.
Update 2
Show create table gives this
CREATE TABLE `productInfo` (
`pID` int(11) NOT NULL AUTO_INCREMENT,
`pOperation` varchar(40) CHARACTER SET latin1 DEFAULT NULL,
`year` year(4) DEFAULT NULL,
`season` varchar(10) CHARACTER SET latin1 DEFAULT NULL,
`pName` varchar(40) CHARACTER SET latin1 DEFAULT NULL,
`category` varchar(40) CHARACTER SET latin1 DEFAULT NULL,
`margin1` text CHARACTER SET latin1,
`margin2` text CHARACTER SET latin1,
PRIMARY KEY (`pID`)
) ENGINE=InnoDB AUTO_INCREMENT=12 DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci
just see that
DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci
But now see that the query
SELECT character_set_name, collation_name
FROM information_schema.columns
WHERE table_schema = 'trac_data'
AND table_name = 'productInfo'
AND column_name = 'pOperation';
gives
character_set_name collation_name
'latin1' 'latin1_swedish_ci'
Thats weird !
Update 3
SELECT hex(pOperation),pOperation FROM trac_data.productInfo;
gave 3F3F3F3F3F which is hex code for actual '?' and not any japanese character so that means no japanese characters are being stored
You have a mix of charsets in your table structure. The table itself uses utf8, but the column in question uses latin 1. You have it defined that way. As long as you have an own charset for your column you can change the table's or the schema's column a thousand times. It won't have any effect on your column. So, instead change the column's charset to either default (to use that of the table) or make it using utf8 explicitely.
When you alter the column's charset existing data will be converted (if possible). Your wrong input however stays wrong, so you have to fill the data again.
Ok i found the cause
CREATE TABLE `productInfo` (
`pID` int(11) NOT NULL AUTO_INCREMENT,
`pOperation` varchar(40) CHARACTER SET latin1 DEFAULT NULL,
`year` year(4) DEFAULT NULL,
`season` varchar(10) CHARACTER SET latin1 DEFAULT NULL,
`pName` varchar(40) CHARACTER SET latin1 DEFAULT NULL,
`category` varchar(40) CHARACTER SET latin1 DEFAULT NULL,
`margin1` text CHARACTER SET latin1,
`margin2` text CHARACTER SET latin1,
PRIMARY KEY (`pID`)
) ENGINE=InnoDB AUTO_INCREMENT=12 DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci
I noticed how in front of each column SET latin1 was present.
So I Just changed to sjis and problem solved.
You have to set the database collation to UTF-8, not only the table collation :
Here is the SQL script result :
I'm writing a webapp in PHP where a list of users can be voted on. The query I'm using to pull a user from the table is quite slow. I suspect there is a much more efficient way to check if the target user has already been voted on by the active user.
SELECT *
FROM users
WHERE id NOT IN (
SELECT ratedid
FROM votes
WHERE who LIKE 12707264
)
AND picture1 NOT LIKE ''
AND cp1 < '10'
AND gender NOT LIKE 'male'
ORDER BY RAND( )
LIMIT 1
Table data as follows:
>SET SQL_MODE = "NO_AUTO_VALUE_ON_ZERO";
>SET time_zone = "+00:00";
/*!40101 SET #OLD_CHARACTER_SET_CLIENT=##CHARACTER_SET_CLIENT */;
/*!40101 SET #OLD_CHARACTER_SET_RESULTS=##CHARACTER_SET_RESULTS */;
/*!40101 SET #OLD_COLLATION_CONNECTION=##COLLATION_CONNECTION */;
/*!40101 SET NAMES utf8 */;
--
-- Database: `notchus_userdb`
--
-- --------------------------------------------------------
--
-- Table structure for table `users`
--
CREATE TABLE IF NOT EXISTS `users` (
`id` bigint(20) NOT NULL,
`username` varchar(30) CHARACTER SET utf8 COLLATE utf8_unicode_ci NOT NULL,
`first_name` varchar(50) CHARACTER SET utf8 COLLATE utf8_unicode_ci NOT NULL,
`last_name` varchar(50) CHARACTER SET utf8 COLLATE utf8_unicode_ci NOT NULL,
`bio` text CHARACTER SET utf8 COLLATE utf8_unicode_ci NOT NULL,
`picture1` varchar(200) CHARACTER SET utf8 COLLATE utf8_unicode_ci NOT NULL,
`cp1` int(100) DEFAULT NULL,
`picture2` varchar(200) CHARACTER SET utf8 COLLATE utf8_unicode_ci NOT NULL,
`picture3` varchar(200) CHARACTER SET utf8 COLLATE utf8_unicode_ci NOT NULL,
`friends` blob,
`relationship_status` varchar(20) CHARACTER SET utf8 COLLATE utf8_unicode_ci NOT NULL,
`relationship_interest` varchar(10) CHARACTER SET utf8 COLLATE utf8_unicode_ci NOT NULL,
`verified` int(1) NOT NULL,
`gender` varchar(10) CHARACTER SET utf8 COLLATE utf8_unicode_ci NOT NULL,
`birthday` varchar(10) CHARACTER SET utf8 COLLATE utf8_unicode_ci NOT NULL,
`hometown` varchar(50) CHARACTER SET utf8 COLLATE utf8_unicode_ci NOT NULL,
`citylocation` varchar(50) CHARACTER SET utf8 COLLATE utf8_unicode_ci NOT NULL,
`oauth_provider` varchar(10) CHARACTER SET utf8 COLLATE utf8_unicode_ci NOT NULL,
`oauth_uid` int(11) NOT NULL,
`ratchet` int(1) DEFAULT NULL,
`boss` int(1) DEFAULT NULL,
`isadmin` int(1) DEFAULT NULL,
`views` int(10) NOT NULL,
`reviews` int(10) NOT NULL,
`email` varchar(50) CHARACTER SET utf8 COLLATE utf8_unicode_ci NOT NULL,
`isuser` int(10) DEFAULT NULL,
PRIMARY KEY (`id`),
KEY `username` (`username`)
) ENGINE=MyISAM DEFAULT CHARSET=latin1;
Server version: 5.1.70-cll
-- PHP Version: 5.3.17
SET SQL_MODE = "NO_AUTO_VALUE_ON_ZERO";
SET time_zone = "+00:00";
/*!40101 SET #OLD_CHARACTER_SET_CLIENT=##CHARACTER_SET_CLIENT */;
/*!40101 SET #OLD_CHARACTER_SET_RESULTS=##CHARACTER_SET_RESULTS */;
/*!40101 SET #OLD_COLLATION_CONNECTION=##COLLATION_CONNECTION */;
/*!40101 SET NAMES utf8 */;
--
-- Database: `notchus_userdb`
--
-- --------------------------------------------------------
--
-- Table structure for table `votes`
--
CREATE TABLE IF NOT EXISTS `votes` (
`uqid` int(11) NOT NULL AUTO_INCREMENT,
`value` tinyint(4) NOT NULL,
`picture` int(11) DEFAULT NULL,
`ratedid` bigint(20) DEFAULT NULL,
`comment` int(11) DEFAULT NULL,
`quote` int(11) DEFAULT NULL,
`who` bigint(20) NOT NULL,
`votedate` int(11) NOT NULL,
`control` varchar(100) NOT NULL,
PRIMARY KEY (`uqid`)
) ENGINE=MyISAM DEFAULT CHARSET=utf8 AUTO_INCREMENT=1084 ;
First of all, LIKE is only for doing wildcard matching. I'm surprised that you're not getting an error doing a LIKE on an integer. Then again, nothing MySQL does surprises me.
SELECT *
FROM users
WHERE id NOT IN (
SELECT ratedid
FROM votes
WHERE who = 12707264
)
AND picture1 <> ''
AND cp1 < '10'
AND gender <> 'male'
ORDER BY RAND( )
LIMIT 1
Also note that if it's OK for picture1 to be an empty string, then you shouldn't have a NOT NULL on the column, and you should store a NULL in the column where is no picture.
Your like statements don't appear to be using wildcards, I see no reason to use them here. Try changing them to equal signs. Indexing your tables could help you as well.
You don't appear to have indices on the foreign key relationships.
try:
create index who on votes (who);
create index q1 on users (picture1, cp1, gender);
Also, you appear to be confused about "like" and "=" - when comparing an integer column (e.g. CP1), use cp1 < 1 or who = 12707264.