MySQL 8 ignoring integer lengths - mysql

I have a MySQL 8.0.19 running in a Docker container and using the InnoDB engine. I have noticed that table integer field lengths are getting ignored.
The issue occurs with integer datatypes regardless if running a CREATE or ALTER query
CREATE TABLE `test` (
`id` int DEFAULT NULL,
`text_field` varchar(20) COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`decimal_field` decimal(6,2) DEFAULT NULL,
`int_field` int DEFAULT NULL
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
The lengths are showing as 0 in my MySQL client (Navicat), but the same occurs if checking in the console with SHOW FULL COLUMNS FROMtest;
mysql> SHOW FULL COLUMNS FROM `test`;
+---------------+--------------+--------------------+------+-----+---------+-------+---------------------------------+---------+
| Field | Type | Collation | Null | Key | Default | Extra | Privileges | Comment |
+---------------+--------------+--------------------+------+-----+---------+-------+---------------------------------+---------+
| id | int | NULL | YES | | NULL | | select,insert,update,references | |
| text_field | varchar(20) | utf8mb4_unicode_ci | YES | | NULL | | select,insert,update,references | |
| decimal_field | decimal(6,2) | NULL | YES | | NULL | | select,insert,update,references | |
| int_field | int | NULL | YES | | NULL | | select,insert,update,references | |
+---------------+--------------+--------------------+------+-----+---------+-------+---------------------------------+---------+
The Type column should be showing int(11) for the two integer fields, but it isn't.
Is this related to something in my MySQL settings, and if so, which variable would have to be changed?

This is a change documented in the MySQL 8.0.19 release notes:
Display width specification for integer data types was deprecated in
MySQL 8.0.17, and now statements that include data type definitions in
their output no longer show the display width for integer types, with
these exceptions:
The type is TINYINT(1). MySQL Connectors make the assumption that
TINYINT(1) columns originated as BOOLEAN columns; this exception
enables them to continue to make that assumption.
The type includes the ZEROFILL attribute.
This change applies to tables, views, and stored routines, and affects
the output from SHOW CREATE and DESCRIBE statements, and from
INFORMATION_SCHEMA tables.
For DESCRIBE statements and INFORMATION_SCHEMA queries, output is
unaffected for objects created in previous MySQL 8.0 versions because
information already stored in the data dictionary remains unchanged.
This exception does not apply for upgrades from MySQL 5.7 to 8.0, for
which all data dictionary information is re-created such that data
type definitions do not include display width. (Bug #30556657, Bug #97680)
The "length" of an integer column doesn't mean anything. A column of int(11) is the same as int(2) or int(40). They are all a fixed-size, 32-bit integer data type. They support the same minimum and maximum value.
The "length" of integer columns has been a confusing feature of MySQL for years. It's only a hint that affects the display width, not the storage or the range of values. Practically, it only matters when you use the ZEROFILL option.
mysql> create table t ( i1 int(6) zerofill, i2 int(12) zerofill );
Query OK, 0 rows affected (0.02 sec)
mysql> insert into t set i1 = 123, i2 = 123;
Query OK, 1 row affected (0.00 sec)
mysql> select * from t;
+--------+--------------+
| i1 | i2 |
+--------+--------------+
| 000123 | 000000000123 |
+--------+--------------+
1 row in set (0.00 sec)
So it's a good thing that the misleading integer "length" is now deprecated and removed. It has caused confusion for many years.

I can confirm that having upgraded AWS RDS to MySQL 8.0.19 that you can now sync using Navicat correctly.
However, PLEASE BE AWARE!!
When updating the id column, if auto_increment is set, Navicat removes auto_increment to change the length and then re-applies it at the end. This causes the auto_increment column to reassign the ids is sequencial order!
ALTER TABLE `database`.`table` MODIFY COLUMN `id` mediumint(0) NOT NULL FIRST;
...
...
ALTER TABLE `database`.`table` MODIFY COLUMN `id` mediumint(0) NOT NULL AUTO_INCREMENT;
If you are using table relationships and do not have the foreign keys setup properly, this will break your database!
Also, if you have id numbers of zero or below in your auto_increment column, this will cause the following error:
Result: 1062 - ALTER TABLE causes auto_increment resequencing,
resulting in duplicate entry '1' for key 'table.PRIMARY'
To avoid the above, you will need to manually change each tables id length to 0 and then save the changes before attempting to use the Navicat sync feature. When saving the changes using Navicat this will automatically change any other int column lengths to 0.
Please ensure you throughly test your changes on a copy of the database before trying to apply to any production databases.

Related

Column declared as NVARCHAR gets created as VARCHAR in MySQL. Both VARCHAR AND NVARCHAR declaration can store non latin characters

I am unable to create NVARCHAR data type in MySQL.
I have the following query -
CREATE TABLE table1 ( column1 NVARCHAR(10) );
This is supposed to create column1 that stores data type NVARCHAR(10). But the query -
DESCRIBE table1;
gives me the output -
+---------+-------------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+---------+-------------+------+-----+---------+-------+
| column1 | varchar(10) | YES | | NULL | |
+---------+-------------+------+-----+---------+-------+
Thus instead of column1 that can store NVARCHAR(10) data type, column1 that can store VARCHAR(10) data type gets created.
Now only NVARCHAR data type is supposed to store non Latin characters.
But the query -
INSERT INTO table1 VALUES ("भारत");
Runs successfully without any error. Here "भारत" is a Hindi word in Devanagari script which in English sounds "Bharat" and translates to "India".
The query -
SELECT * FROM table1;
gives display as expected -
+--------------+
| column1 |
+--------------+
| भारत |
+--------------+
I guess may be MySQL treats VARCHAR internally as NVARCHAR. But I can't find any documentation stating so.
The following is a link from MySQL developers website -
https://dev.mysql.com/doc/refman/8.0/en/charset-national.html
Here it says that NVARCHAR is fully supported.
To find out if non Latin characters can be stored in a column defined as VARCHAR I ran the following queries -
CREATE TABLE table2 ( column2 VARCHAR(10) );
DESCRIBE table2;
This gives me the output -
+---------+-------------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+---------+-------------+------+-----+---------+-------+
| column2 | varchar(10) | YES | | NULL | |
+---------+-------------+------+-----+---------+-------+
Here column2 that can store VARCHAR(10) data type gets created as expected.
Running the query -
INSERT INTO table2 VALUES ("भारत");
runs without any error.
and the query -
SELECT * FROM table2;
gives expected output -
+--------------+
| column2 |
+--------------+
| भारत |
+--------------+
Thus even if I declare column2 as VARCHAR(10) I can successfully store non Latin characters (here Devanagari characters of Hindi language).
The most logical conclusion is that regardless of declaring a column as VARCHAR or NVARCHAR MySQL always internally stores it as NVARCHAR. But I can't find any documentation regarding the same.
The following stackoverflow question gets closest to my question -
Issue Converting varchar to nvarchar mysql
But there is no answer provided to the question.
I am using operating system Ubuntu 20.04 and MySQL version - 8.0.26
Which information you can save is stored in character set and collation.
so as the default is utf8, bith can save hindi or chines or kisuali in their 4 byites
but
CREATE TABLE table1 ( column1 NVARCHAR(10),column2 VARCHAR(10) );
Actually is treated slightly different
CREATE TABLE `table1` (
`column1` varchar(10) CHARACTER SET utf8 COLLATE utf8_general_ci DEFAULT NULL,
`column2` varchar(10) DEFAULT NULL
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_0900_ai_ci
in the sample database the Default is
DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_0900_ai_ci
But the national varchar is like the standard defines
CHARACTER SET utf8 COLLATE utf8_general_ci
For your hindi word "भारत" it makes no differenz, but for some charachters there can be "problems"
Get in the habit of using SHOW CREATE TABLE instead of DESCRIBE. It would have answered your question.
mysql> CREATE TABLE nv ( column1 NVARCHAR(10) );
Query OK, 0 rows affected, 1 warning (0.05 sec)
mysql> show warnings;
+---------+------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Level | Code | Message |
+---------+------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Warning | 3720 | NATIONAL/NCHAR/NVARCHAR implies the character set UTF8MB3, which will be replaced by UTF8MB4 in a future release. Please consider using CHAR(x) CHARACTER SET UTF8MB4 in order to be unambiguous. |
+---------+------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
1 row in set (0.00 sec)
mysql> SHOW CREATE TABLE nv\G
*************************** 1. row ***************************
Table: nv
Create Table: CREATE TABLE `nv` (
`column1` varchar(10) CHARACTER SET utf8 COLLATE utf8_general_ci DEFAULT NULL
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_520_ci
1 row in set (0.00 sec)
The Warning gives you a hint of an important problem, should you ever try to store Chinese or Emoji in the column. utf8mb4 is needed.
So, you should say
CREATE TABLE nv ( column1 VARCHAR(10) CHARACTER SET utf8mb4 );
That is, don't use NVARCHAR, use VARCHAR and specify the appropriate character set.
utf8 happens to be OK for DEVANAGARI, as in your example.

What does ENGINE means in MySQL?

I have a simple 'people' table:
mysql> desc people;
+-------+--------------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+-------+--------------+------+-----+---------+-------+
| id | int | NO | PRI | NULL | |
| name | varchar(255) | YES | | NULL | |
+-------+--------------+------+-----+---------+-------+
2 rows in set (0.01 sec)
When I do show create table people; command, it shows me whole command that created my table:
mysql> show create table people;
+--------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Table | Create Table |
+--------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| people | CREATE TABLE `people` (
`id` int NOT NULL,
`name` varchar(255) DEFAULT NULL,
PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_0900_ai_ci |
+--------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
1 row in set (0.00 sec)
In this command, I see a sentence ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_0900_ai_ci.
Question is: What does it mean? Especially, what does ENGINE=InnoDB mean?
InnoDB is a storage engine in MySQL. As explained in the documentation:
Storage engines are MySQL components that handle the SQL operations for different table types. InnoDB is the default and most general-purpose storage engine, and Oracle recommends using it for tables except for specialized use cases. (The CREATE TABLE statement in MySQL 8.0 creates InnoDB tables by default.)
MySQL Server uses a pluggable storage engine architecture that enables storage engines to be loaded into and unloaded from a running MySQL server.
As of the current state of art, MyISAM, that was the only available engine in the early 5.x days, is being abandonned and InnoDB is the de facto standard: it offers most typical functionalities that can be find in other high-profile databases, such as transactions and row-level locking.
InnoDB it's a storage engin that allow Foriegn Key ,Rollback , commit and ACID Compliant...
CHARSET = is a set of characters that are legal in a string.
uft8mb4 means that each character is stored as a maximum of 4 bytes in the UTF-8 encoding scheme.
ai refers accent insensitivity
ci refers to case insensitivity.
COLLATE = set of rules used to compare characters

MySQL self-join takes too long—can I streamline this query?

I'm hoping (and pretty sure that) someone out there is much better at MySQL queries than msyelf.
I have a query which checks a table that contains information on :
- a search term
- title and price results from various sites using this search term
For the sake of streamlining, I've inserted the data already converted to lowercase with spaces removed and the whole thing trimmed to 11 characters to help reduce the load on the MySQL server.
The query is designed to find the maximum cost and minimum cost of likely equal titles and determine a price difference if it exists.
Having read some similar questions here, I've also prepended EXPLAIN EXTENDED to the query to see if that would help and I'm including the results along with the query.
The query as is :
SELECT
a.pricesrch11,
b.pricesrch11,
a.pricegroup11,
b.pricegroup11,
a.priceamt - b.priceamt AS pricediff
FROM ebssavings a
LEFT JOIN ebssavings b ON ( a.pricesrch11 = b.pricesrch11 )
AND (a.pricegroup11 = a.pricesrch11)
AND (b.pricegroup11 = a.pricesrch11)
WHERE a.priceamt - b.priceamt >0
GROUP BY a.pricesrch11
The results of the EXPLAIN :
select_type | table | type | possible_keys | key | key_len | ref | rows | Extra
1 | SIMPLE | a | ALL | pricesrch11,pricegroup11 | NULL | NULL | NULL | 8816 | Using where; Using temporary; Using filesort
1 | SIMPLE | b | ALL | pricesrch11,pricegroup11 | NULL | NULL | NULL | 6612 | Using where
ADDENDUM :
I just ran this query and got the following result :
Showing rows 0 - 4 ( 5 total, Query took 66.8119 sec)
CREATE TABLE IF NOT EXISTS ebssavings
( priceid int(44) NOT NULL auto_increment,
priceamt decimal(10,2) NOT NULL,
pricesrch11 varchar(11) character set utf8 collate utf8_unicode_ci NOT NULL,
pricegroup11 varchar(11) character set utf8 collate utf8_unicode_ci NOT NULL,
pricedate timestamp NOT NULL default CURRENT_TIMESTAMP,
PRIMARY KEY (priceid),
KEY priceamt (priceamt),
KEY pricesrch11 (pricesrch11),
KEY pricegroup11 (pricegroup11) )
ENGINE=MyISAM DEFAULT CHARSET=latin1 AUTO_INCREMENT=8817
MORE INFO ON THE NEW INDEXES (removed pricegroup11, and made a composite index called srchandtitle from pricesrch11 and pricegroup11):
Edit Drop PRIMARY BTREE Yes No priceid 169 A
Edit Drop priceamt BTREE No No priceamt 56 A
Edit Drop pricesrch11 BTREE No No pricesrch11 12 A
Edit Drop srchandtitle BTREE No No pricesrch11 12 A
pricegroup11 169 A
Create two indexes:
PriceSrch11
A clustered index on pricesrch11,pricegroup11
Remove the Key on pricegroup11 and add a composite clustered key on pricesrch11,pricegroup11.
Also move the table to InnoDB.
It seems that things have sped up now with the changes made to the table and the indexes.
I've emptied the table and am beginning again.
Thank you all for your help.
-A

Mysql VARCHAR to CHAR conversion behavior question

I'm looking at this Mysql question link:
And I can't repeat the behavior described in the answer to that question.
I tried creating tables with CHAR and VARCHAR column of various lengths and it doesn't matter what length - SHOW CREATE TABLE always return the data type that I've originally defined.
So - no CHAR->VARCHAR switching is going on.
Is answer to the question below only partially correct (I'm talking only about items 1 and 2)?
Q:
When you create a table, and then run SHOW CREATE TABLE on it, you occasionally get different results than what you typed in. What does MySQL modify in your newly created tables?
A (supposedly):
VARCHARs with length less than 4 become CHARs
CHARs with length more than 3 become VARCHARs.
NOT NULL gets added to the columns declared as PRIMARY KEYs
Default values such as NULL are specified for each column
There is a page in the MySQL's manual that answers some of your questions : 12.1.14.2. Silent Column Specification Changes.
Quoting some portions that correspond to items you posted in your question :
For item 3 :
Columns that are part of a PRIMARY
KEY are made NOT NULL even if not
declared that way.
About the size of varchar columns (not exactly one of your items, though) :
If strict SQL mode is not enabled, a
VARCHAR column with a length
specification greater than 65535 is
converted to TEXT, and a VARBINARY
column with a length specification
greater than 65535 is converted to
BLOB. Otherwise, an error occurs in
either of these cases.
And that page ends with the following sentence :
To see whether MySQL used a data type
other than the one you specified,
issue a DESCRIBE or SHOW CREATE
TABLE statement after creating or
altering the table.
So I'm guessing you might expect some additional differences, that are not listed.
Doing a quick test, here's a create table statement :
create table test_2 (
id int primary key,
blah_vc varchar(2),
blah_c char(5)
) engine=InnoDb;
And the table that's created gives :
mysql> desc test_2;
+---------+------------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+---------+------------+------+-----+---------+-------+
| id | int(11) | NO | PRI | NULL | |
| blah_vc | varchar(2) | YES | | NULL | |
| blah_c | char(5) | YES | | NULL | |
+---------+------------+------+-----+---------+-------+
3 rows in set (0.00 sec)
mysql> show create table test_2;
+--------+--------------------------------------------+
| Table | Create Table |
+--------+--------------------------------------------+
| test_2 | CREATE TABLE `test_2` (
`id` int(11) NOT NULL,
`blah_vc` varchar(2) DEFAULT NULL,
`blah_c` char(5) DEFAULT NULL,
PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1 |
+--------+--------------------------------------------+
1 row in set (0.00 sec)
So :
no : varchar has not been transformed to char
no : char has not been transformed to varchar
yes : not null has been added to primary key
well, that one is funny : describe says it hasn't,
but show create table indicates it has...
Anyway : it makes sense, for a primary key column, to not be nullable.
yes : null is specified as default for columns that can be null.
That question is pretty old, written back in the days of MySQL 4.
As of MySQL 5.0, 1 & 2 from that list are no longer true.

MySQL database for hashes

I'm currently trying to create a mySQL database that holds hashes such as MD5 hashes. I'm using PHPmyAdmin version 3.3.9, and MySQL client version: 4.1.22
I already created a database named hashes. I'm new to mySQL so how can I add a table with data for a hash?
Hash column should be a CHAR(32) as that is the length of the hash:
CREATE TABLE `hashes` (
`id` INT NOT NULL AUTO_INCREMENT,
`hash` CHAR(32),
PRIMARY KEY (`id`)
);
mysql> describe hashes;
+-------+----------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+-------+----------+------+-----+---------+----------------+
| id | int(11) | NO | PRI | NULL | auto_increment |
| hash | char(32) | YES | | NULL | |
+-------+----------+------+-----+---------+----------------+
If you want to select from the table given user input:
-- Insert sample data:
mysql> INSERT INTO `hashes` VALUES (null, MD5('hello'));
Query OK, 1 row affected (0.00 sec)
-- Test retrieval:
mysql> SELECT * FROM `hashes` WHERE `hash` = MD5('hello');
+----+----------------------------------+
| id | hash |
+----+----------------------------------+
| 1 | 5d41402abc4b2a76b9719d911017c592 |
+----+----------------------------------+
1 row in set (0.00 sec)
You can add a key on hash for better performance.
A hash can be stored as binary data or (more convenient) as text. The PHP MD5 function by default outputs a string of 32 characters, so you will need a table with a field that can hold a string of 32 character. That's all. :)
Take a look here:
http://dev.mysql.com/doc/refman/5.1/en/create-table.html
You can do it using phpMyAdmin of course but it is better to understand the actual SQL.
If you open the database (click it's name in the left hand column), there will be a "Create new table on database dbname" section. Enter the name of the table you want (e.g. hashes), and the number of fields you want.
This is where we need a bit more information: if you just want to store hashes and hashes alone then you'll need two fields: a unique hash ID (of type integer, with auto_increment set to true) and a 32 character text field. I'm a little unsure of the utility of this, so if you post a bit more information we might be able to help you out with what you're trying to achieve a little better?
Edit:
In that case, you'll need three fields: a hash id (a unique reference for each entry in the table), the hash, and the plaintext to which it correlates. Set the hash id to be an integer, with auto_increment set to true. Set the field for the hash to be varchar of length 32, and the plaintext field to 'text'.