Mysql VARCHAR to CHAR conversion behavior question - mysql

I'm looking at this Mysql question link:
And I can't repeat the behavior described in the answer to that question.
I tried creating tables with CHAR and VARCHAR column of various lengths and it doesn't matter what length - SHOW CREATE TABLE always return the data type that I've originally defined.
So - no CHAR->VARCHAR switching is going on.
Is answer to the question below only partially correct (I'm talking only about items 1 and 2)?
Q:
When you create a table, and then run SHOW CREATE TABLE on it, you occasionally get different results than what you typed in. What does MySQL modify in your newly created tables?
A (supposedly):
VARCHARs with length less than 4 become CHARs
CHARs with length more than 3 become VARCHARs.
NOT NULL gets added to the columns declared as PRIMARY KEYs
Default values such as NULL are specified for each column

There is a page in the MySQL's manual that answers some of your questions : 12.1.14.2. Silent Column Specification Changes.
Quoting some portions that correspond to items you posted in your question :
For item 3 :
Columns that are part of a PRIMARY
KEY are made NOT NULL even if not
declared that way.
About the size of varchar columns (not exactly one of your items, though) :
If strict SQL mode is not enabled, a
VARCHAR column with a length
specification greater than 65535 is
converted to TEXT, and a VARBINARY
column with a length specification
greater than 65535 is converted to
BLOB. Otherwise, an error occurs in
either of these cases.
And that page ends with the following sentence :
To see whether MySQL used a data type
other than the one you specified,
issue a DESCRIBE or SHOW CREATE
TABLE statement after creating or
altering the table.
So I'm guessing you might expect some additional differences, that are not listed.
Doing a quick test, here's a create table statement :
create table test_2 (
id int primary key,
blah_vc varchar(2),
blah_c char(5)
) engine=InnoDb;
And the table that's created gives :
mysql> desc test_2;
+---------+------------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+---------+------------+------+-----+---------+-------+
| id | int(11) | NO | PRI | NULL | |
| blah_vc | varchar(2) | YES | | NULL | |
| blah_c | char(5) | YES | | NULL | |
+---------+------------+------+-----+---------+-------+
3 rows in set (0.00 sec)
mysql> show create table test_2;
+--------+--------------------------------------------+
| Table | Create Table |
+--------+--------------------------------------------+
| test_2 | CREATE TABLE `test_2` (
`id` int(11) NOT NULL,
`blah_vc` varchar(2) DEFAULT NULL,
`blah_c` char(5) DEFAULT NULL,
PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1 |
+--------+--------------------------------------------+
1 row in set (0.00 sec)
So :
no : varchar has not been transformed to char
no : char has not been transformed to varchar
yes : not null has been added to primary key
well, that one is funny : describe says it hasn't,
but show create table indicates it has...
Anyway : it makes sense, for a primary key column, to not be nullable.
yes : null is specified as default for columns that can be null.

That question is pretty old, written back in the days of MySQL 4.
As of MySQL 5.0, 1 & 2 from that list are no longer true.

Related

MySQL 8 ignoring integer lengths

I have a MySQL 8.0.19 running in a Docker container and using the InnoDB engine. I have noticed that table integer field lengths are getting ignored.
The issue occurs with integer datatypes regardless if running a CREATE or ALTER query
CREATE TABLE `test` (
`id` int DEFAULT NULL,
`text_field` varchar(20) COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`decimal_field` decimal(6,2) DEFAULT NULL,
`int_field` int DEFAULT NULL
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
The lengths are showing as 0 in my MySQL client (Navicat), but the same occurs if checking in the console with SHOW FULL COLUMNS FROMtest;
mysql> SHOW FULL COLUMNS FROM `test`;
+---------------+--------------+--------------------+------+-----+---------+-------+---------------------------------+---------+
| Field | Type | Collation | Null | Key | Default | Extra | Privileges | Comment |
+---------------+--------------+--------------------+------+-----+---------+-------+---------------------------------+---------+
| id | int | NULL | YES | | NULL | | select,insert,update,references | |
| text_field | varchar(20) | utf8mb4_unicode_ci | YES | | NULL | | select,insert,update,references | |
| decimal_field | decimal(6,2) | NULL | YES | | NULL | | select,insert,update,references | |
| int_field | int | NULL | YES | | NULL | | select,insert,update,references | |
+---------------+--------------+--------------------+------+-----+---------+-------+---------------------------------+---------+
The Type column should be showing int(11) for the two integer fields, but it isn't.
Is this related to something in my MySQL settings, and if so, which variable would have to be changed?
This is a change documented in the MySQL 8.0.19 release notes:
Display width specification for integer data types was deprecated in
MySQL 8.0.17, and now statements that include data type definitions in
their output no longer show the display width for integer types, with
these exceptions:
The type is TINYINT(1). MySQL Connectors make the assumption that
TINYINT(1) columns originated as BOOLEAN columns; this exception
enables them to continue to make that assumption.
The type includes the ZEROFILL attribute.
This change applies to tables, views, and stored routines, and affects
the output from SHOW CREATE and DESCRIBE statements, and from
INFORMATION_SCHEMA tables.
For DESCRIBE statements and INFORMATION_SCHEMA queries, output is
unaffected for objects created in previous MySQL 8.0 versions because
information already stored in the data dictionary remains unchanged.
This exception does not apply for upgrades from MySQL 5.7 to 8.0, for
which all data dictionary information is re-created such that data
type definitions do not include display width. (Bug #30556657, Bug #97680)
The "length" of an integer column doesn't mean anything. A column of int(11) is the same as int(2) or int(40). They are all a fixed-size, 32-bit integer data type. They support the same minimum and maximum value.
The "length" of integer columns has been a confusing feature of MySQL for years. It's only a hint that affects the display width, not the storage or the range of values. Practically, it only matters when you use the ZEROFILL option.
mysql> create table t ( i1 int(6) zerofill, i2 int(12) zerofill );
Query OK, 0 rows affected (0.02 sec)
mysql> insert into t set i1 = 123, i2 = 123;
Query OK, 1 row affected (0.00 sec)
mysql> select * from t;
+--------+--------------+
| i1 | i2 |
+--------+--------------+
| 000123 | 000000000123 |
+--------+--------------+
1 row in set (0.00 sec)
So it's a good thing that the misleading integer "length" is now deprecated and removed. It has caused confusion for many years.
I can confirm that having upgraded AWS RDS to MySQL 8.0.19 that you can now sync using Navicat correctly.
However, PLEASE BE AWARE!!
When updating the id column, if auto_increment is set, Navicat removes auto_increment to change the length and then re-applies it at the end. This causes the auto_increment column to reassign the ids is sequencial order!
ALTER TABLE `database`.`table` MODIFY COLUMN `id` mediumint(0) NOT NULL FIRST;
...
...
ALTER TABLE `database`.`table` MODIFY COLUMN `id` mediumint(0) NOT NULL AUTO_INCREMENT;
If you are using table relationships and do not have the foreign keys setup properly, this will break your database!
Also, if you have id numbers of zero or below in your auto_increment column, this will cause the following error:
Result: 1062 - ALTER TABLE causes auto_increment resequencing,
resulting in duplicate entry '1' for key 'table.PRIMARY'
To avoid the above, you will need to manually change each tables id length to 0 and then save the changes before attempting to use the Navicat sync feature. When saving the changes using Navicat this will automatically change any other int column lengths to 0.
Please ensure you throughly test your changes on a copy of the database before trying to apply to any production databases.

Simple Sphinx & mySQL Query

Forgive me for asking what should be a simple question but I am totally new to Sphinx.
I am using Sphinx with a mySQL datastore. The table looks like below with the Title and Content fields indexed by Sphinx.
CREATE TABLE `documents` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`group_id` int(11) NOT NULL,
`group_id2` int(11) NOT NULL,
`date_added` datetime NOT NULL,
`title` varchar(255) NOT NULL,
`content` text NOT NULL,
`url` varchar(255) NOT NULL,
`links` int(11) NOT NULL,
`hosts` int(11) NOT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `url` (`url`)
) ENGINE=InnoDB AUTO_INCREMENT=439043 DEFAULT CHARSET=latin1
Now, if I connect through Sphinx with
mysql -h0 -P9306
I can run a simple query like...
SELECT * FROM test1 WHERE MATCH('test document');
And I will get back a result set like...
+--------+----------+------------+
| id | group_id | date_added |
+--------+----------+------------+
| 360625 | 1 | 1499727792 |
| 362257 | 1 | 1499727807 |
| 362777 | 1 | 1499727811 |
| 159717 | 1 | 1499717614 |
| 160557 | 1 | 1499717621 |
----------------------------------
When what I actually want is it to return a result set with column values from the documents table (like the URL, Title, Links, Hosts, etc. columns) and, if all possible, sort these by the relevancy of the Sphinx match.
Can that be accomplished in a single query? What might it look like?
Thanks in advance!
Two (main) options
Take the ids from the SphinxQL result, and run a MySQL Query to get the full details, see http://sphinxsearch.com/info/faq/#row-storage
eg SELECT * FROM documents WHERE id IN (3,5,7) ORDER BY FIELD(id,3,5,7)
This MySQL query, should be VERY quick, because its a PK lookup, and only retrieving a few rows (ie one page of results) - the heavy lifting of searching the whole table has already been done in first Sphinx Query.
Duplicate all the columns you want to retrieve in the resultset as Attributes. You've already made group_id and date_added as attributes, would need to make more attributes.
sql_field_string is a very convenient shortcut to make BOTH a Field and an String Attribute from one column. Not available for other column types, but less useful, as numeric columns, are not typically needed as fields anyway.
option 1 is good in it avoids duplicating the data, and saves memory (Sphinx wants to typically hold attributes in memory) - and may be most practical on big datasets.
whereas option 2 is good in that it avoids a second query for each result. But because have a copy of data, it may mean additional complication syncing.
Doesn't look relevant in your case, but if say had a 'clicks' column, which you want in increment often (when users click!), and need it in resultset but you don't really need it in sphinx for query purposes, the first option, would allow you only have to increment it in database, and the mysql query would always get the live value. But the second option means having to keep sphinx index in 'sync' at all times)

MySQL update query not using indexed columns

I tried the following SQL query to update the table INDEXED_MERCHANT where I have 10000 records in the table. I indexed both "Name" and "A" as Indexed keys to improve the update query performance. By executing the command SHOW CREATE TABLE INDEXED_MERCHANT; I'll get an output result as follows:
INDEXED_MERCHANT | CREATE TABLE `INDEXED_MERCHANT` (
`ID` varchar(50) NOT NULL,
`NAME` varchar(200) DEFAULT NULL,
`ONLINE_STATUS` varchar(10) NOT NULL,
`A` varchar(100) DEFAULT NULL,
`B` varchar(200) DEFAULT NULL,
PRIMARY KEY (`ID`),
KEY `NAME` (`NAME`,`A`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 |
Here this means both my keys are recognized as indexed keys. When I execute the following command, the "Extra" column result says the query doesn't using the index keys. How should I achieve my goal ?
Executed query : EXPLAIN EXTENDED UPDATE INDEXED_MERCHANT SET ONLINE_STATUS = '0' WHERE NAME = 'A 205' AND A = 'P 205';
Result:
+----+-------------+------------------+-------+---------------+------+---------+-------------+------+----------+-------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | filtered | **Extra** |
+----+-------------+------------------+-------+---------------+------+---------+-------------+------+----------+-------------+
| 1 | SIMPLE | INDEXED_MERCHANT | range | NAME | NAME | 906 | const,const | 1 | 100.00 | **Using where** |
+----+-------------+------------------+-------+---------------+------+---------+-------------+------+----------+-------------+
Your query is hitting "NAME" index as evident in explain output column "key".
Here is the explanation of key and extra columns from mysql documentation
key
The key column indicates the key (index) that MySQL actually decided to use. If MySQL decides to use one of the possible_keys indexes to look up rows, that index is listed as the key value.
Using where
A WHERE clause is used to restrict which rows to match against the next table or send to the client. Unless you specifically intend to fetch or examine all rows from the table, you may have something wrong in your query if the Extra value is not Using where and the table join type is ALL or index. Even if you are using an index for all parts of a WHERE clause, you may see Using where if the column can be NULL.

MySQL self-join takes too long—can I streamline this query?

I'm hoping (and pretty sure that) someone out there is much better at MySQL queries than msyelf.
I have a query which checks a table that contains information on :
- a search term
- title and price results from various sites using this search term
For the sake of streamlining, I've inserted the data already converted to lowercase with spaces removed and the whole thing trimmed to 11 characters to help reduce the load on the MySQL server.
The query is designed to find the maximum cost and minimum cost of likely equal titles and determine a price difference if it exists.
Having read some similar questions here, I've also prepended EXPLAIN EXTENDED to the query to see if that would help and I'm including the results along with the query.
The query as is :
SELECT
a.pricesrch11,
b.pricesrch11,
a.pricegroup11,
b.pricegroup11,
a.priceamt - b.priceamt AS pricediff
FROM ebssavings a
LEFT JOIN ebssavings b ON ( a.pricesrch11 = b.pricesrch11 )
AND (a.pricegroup11 = a.pricesrch11)
AND (b.pricegroup11 = a.pricesrch11)
WHERE a.priceamt - b.priceamt >0
GROUP BY a.pricesrch11
The results of the EXPLAIN :
select_type | table | type | possible_keys | key | key_len | ref | rows | Extra
1 | SIMPLE | a | ALL | pricesrch11,pricegroup11 | NULL | NULL | NULL | 8816 | Using where; Using temporary; Using filesort
1 | SIMPLE | b | ALL | pricesrch11,pricegroup11 | NULL | NULL | NULL | 6612 | Using where
ADDENDUM :
I just ran this query and got the following result :
Showing rows 0 - 4 ( 5 total, Query took 66.8119 sec)
CREATE TABLE IF NOT EXISTS ebssavings
( priceid int(44) NOT NULL auto_increment,
priceamt decimal(10,2) NOT NULL,
pricesrch11 varchar(11) character set utf8 collate utf8_unicode_ci NOT NULL,
pricegroup11 varchar(11) character set utf8 collate utf8_unicode_ci NOT NULL,
pricedate timestamp NOT NULL default CURRENT_TIMESTAMP,
PRIMARY KEY (priceid),
KEY priceamt (priceamt),
KEY pricesrch11 (pricesrch11),
KEY pricegroup11 (pricegroup11) )
ENGINE=MyISAM DEFAULT CHARSET=latin1 AUTO_INCREMENT=8817
MORE INFO ON THE NEW INDEXES (removed pricegroup11, and made a composite index called srchandtitle from pricesrch11 and pricegroup11):
Edit Drop PRIMARY BTREE Yes No priceid 169 A
Edit Drop priceamt BTREE No No priceamt 56 A
Edit Drop pricesrch11 BTREE No No pricesrch11 12 A
Edit Drop srchandtitle BTREE No No pricesrch11 12 A
pricegroup11 169 A
Create two indexes:
PriceSrch11
A clustered index on pricesrch11,pricegroup11
Remove the Key on pricegroup11 and add a composite clustered key on pricesrch11,pricegroup11.
Also move the table to InnoDB.
It seems that things have sped up now with the changes made to the table and the indexes.
I've emptied the table and am beginning again.
Thank you all for your help.
-A

MySQL database for hashes

I'm currently trying to create a mySQL database that holds hashes such as MD5 hashes. I'm using PHPmyAdmin version 3.3.9, and MySQL client version: 4.1.22
I already created a database named hashes. I'm new to mySQL so how can I add a table with data for a hash?
Hash column should be a CHAR(32) as that is the length of the hash:
CREATE TABLE `hashes` (
`id` INT NOT NULL AUTO_INCREMENT,
`hash` CHAR(32),
PRIMARY KEY (`id`)
);
mysql> describe hashes;
+-------+----------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+-------+----------+------+-----+---------+----------------+
| id | int(11) | NO | PRI | NULL | auto_increment |
| hash | char(32) | YES | | NULL | |
+-------+----------+------+-----+---------+----------------+
If you want to select from the table given user input:
-- Insert sample data:
mysql> INSERT INTO `hashes` VALUES (null, MD5('hello'));
Query OK, 1 row affected (0.00 sec)
-- Test retrieval:
mysql> SELECT * FROM `hashes` WHERE `hash` = MD5('hello');
+----+----------------------------------+
| id | hash |
+----+----------------------------------+
| 1 | 5d41402abc4b2a76b9719d911017c592 |
+----+----------------------------------+
1 row in set (0.00 sec)
You can add a key on hash for better performance.
A hash can be stored as binary data or (more convenient) as text. The PHP MD5 function by default outputs a string of 32 characters, so you will need a table with a field that can hold a string of 32 character. That's all. :)
Take a look here:
http://dev.mysql.com/doc/refman/5.1/en/create-table.html
You can do it using phpMyAdmin of course but it is better to understand the actual SQL.
If you open the database (click it's name in the left hand column), there will be a "Create new table on database dbname" section. Enter the name of the table you want (e.g. hashes), and the number of fields you want.
This is where we need a bit more information: if you just want to store hashes and hashes alone then you'll need two fields: a unique hash ID (of type integer, with auto_increment set to true) and a 32 character text field. I'm a little unsure of the utility of this, so if you post a bit more information we might be able to help you out with what you're trying to achieve a little better?
Edit:
In that case, you'll need three fields: a hash id (a unique reference for each entry in the table), the hash, and the plaintext to which it correlates. Set the hash id to be an integer, with auto_increment set to true. Set the field for the hash to be varchar of length 32, and the plaintext field to 'text'.