If i modify the max number of allowed digits in MYSQL 5.7 from double(8,2) to double(12,2), is the change immediate or will it need to process all rows??
You can test this to see if it can be changed as an instant change:
mysql> create table mytable (id serial primary key, d double(8,2));
mysql> alter table mytable modify column d double(25,2), algorithm=inplace, lock=none;
Query OK, 0 rows affected, 1 warning (0.01 sec)
Records: 0 Duplicates: 0 Warnings: 1
The options algorithm=inplace, lock=none mean you want the alter to run as an instant change, without performing a table copy. By default, MySQL runs the DDL change in that mode if the change can be done in that mode.
If you request it explicitly, but the change cannot be done in that mode, then you'll get an error.
For example:
mysql> alter table mytable modify column d float(8,2), algorithm=inplace, lock=none;
ERROR 1846 (0A000): ALGORITHM=INPLACE is not supported.
Reason: Cannot change column type INPLACE. Try ALGORITHM=COPY.
In this example I'm changing the 8-byte DOUBLE to a 4-byte FLOAT. Any change to the size of a data type cannot be done without copying the table. So the request to do it as an instant change fails and the error shown is returned.
So if you're in doubt about whether a given change can be done instantly, you can use this method to test it. You don't have to do the test against your production table! I did this test on my local instance, without even adding any data to the table. I just created an empty table as I showed above, and ran the DDL.
You should read https://dev.mysql.com/doc/refman/8.0/en/innodb-online-ddl-operations.html for details on which types of DDL changes can be done inplace.
As written in the MySQL Reference Manual all ALTER Table statements needs processing time. On top of this MySQL will convert the data:
For data type changes using CHANGE or MODIFY, MySQL tries to convert existing column values to the new type as well as possible.
Therefore all Columns will be visited.
Beside the fact that specifying number of digits for floating point data types is deprecated, a change from double(8,2) to double(12,2) doesn't change the column type, it is still a double precision 8-byte number, so not even a single row will change it's value.
Example:
CREATE TABLE t1 (a double(6,2));
INSERT INTO t1 VALUES (1234.56);
# change the precision
ALTER TABLE t1 change a a double(4,2);
INSERT INTO t1 VALUES (1234.56);
ERROR 1264 (22003): Out of range value for column 'a' at row 1
SELECT a FROM t1;
+---------+
| a |
+---------+
| 1234.56 |
+---------+
Even if 1234.56 doesn't fit in double(4,2) it is still unchanged.
Related
I have table with:
1) Encrypted_ID varchar (256)
2) Initialization Vector(iv)varchar(256).
I would like to decrypt the column value using the key
I am using:
select Cast(AES_DECRYPT(Encrypted_ID,'Key',InitializationVector_iv)as CHAR ) as DecryptedValue from MyTable;
The result is Null.
I Also tried:
select Cast(AES_DECRYPT(AES_ENCRYPT(Encrypted_ID,'Key',InitializationVector_iv),'Key') as CHAR ) as DecryptedValue from MyTable;
The result is blob for few rows.
I'm not sure what I'm doing wrong here. Can any one help with the syntax to decrypt the column when I have:
Key
Initialization Vector value
Encrypted Column
There's actually nothing wrong with your first query, syntactically it's spot on as this worked example demonstrates.
mysql> SET ##SESSION.block_encryption_mode = 'aes-256-cbc';
mysql> create table MyTable(
-> Encrypted_ID varbinary(256),
-> InitializationVector_iv varbinary(16)
-> );
Query OK, 0 rows affected (0.93 sec)
mysql> SET #iv = RANDOM_BYTES(16);
Query OK, 0 rows affected (0.00 sec)
mysql> INSERT INTO MyTable SET Encrypted_ID = AES_ENCRYPT('hello','key', #iv), InitializationVector_iv = #iv;
Query OK, 1 row affected (0.17 sec)
mysql> SELECT CAST(AES_DECRYPT(Encrypted_ID,'key', InitializationVector_iv) AS CHAR) from MyTable;
+------------------------------------------------------------------------+
| CAST(AES_DECRYPT(Encrypted_ID,'key', InitializationVector_iv) AS CHAR) |
+------------------------------------------------------------------------+
| hello |
+------------------------------------------------------------------------+
1 row in set (0.00 sec)
As for why it's not working, I managed to get the query to return NULL in 2 scenarios. One, you get NULL returned if you use a different iv for encryption and decryption, so you might want to look at how you are storing as the iv. Two, you get NULL where you have the block_encryption_mode variable set differently when storing and trying to retrieve the value, check that you're not accidentally reverting to the default 'aes-128-ebc between sessions. There may be others...
The second query will fail because you need to supply the iv to both of he encryption and decryption functions, you only use it to encrypt. Also, since you are taking the values from the MyTable, Encrypted_ID will already be encrypted and the effect of this query would be to encrypt it again, before reversing that to get you back to the stored (encrypted) value.
Finally, AES is only going to use 16 bytes of the iv so you might as well make that VARBINARY(16).
AES doesn't work with MySQL Workbench in my case.
I have to use the mysql console.
MySql Workbench worked for me.
In my Case, encrypted value was encoded in base 64.
So I had to decode base 64 value and IV Using "From_base64" function.
SET block_encryption_mode = 'aes-256-cbc';
set #k = 'Key';
set #iv = From_base64('InitializationVector');
set #v = from_base64('EncryptedValue');
select CAST(AES_DECRYPT(#v, #k, #iv) AS CHAR);
Please make sure the encryption type, base 64 encoding, Hex/Unhex of the values/Iv are correct before you start working on the decryption.
Review MYSql functions
https://dev.mysql.com/doc/refman/8.0/en/string-functions.html
Hope this helps for someone.
I have a MySQL table t with over 150 million rows. One of the columns (c) was a VARCHAR(64) containing a 64-digit hexadecimal number. To save space and make things faster, I wanted to decode the hex and turn it into a BINARY(32) column.
My plan was to use three queries:
ALTER TABLE t CHANGE c c BINARY(64) NOT NULL;
UPDATE t SET c=UNHEX(c);
ALTER TABLE t CHANGE c c BINARY(32) NOT NULL;
The first 2 worked perfectly, but on the 3rd query I'm getting the error:
#1265 - Data truncated for column 'c' at row 1
I understand that I am truncating data, that's exactly what I want. I want to get rid of the 32 0x00 bytes at the end of the BINARY(64) to turn it into a BINARY(32).
Things I've tried:
UPDATE t SET c=LEFT(c, 32); did not seem to do anything at all.
Using ALTER IGNORE TABLE gives me a syntax error.
To get around the #1265 - Data truncated for column ... error you must remove the STRICT_TRANS_TABLES flag from the global sql_mode variable.
The query SHOW VARIABLES LIKE 'sql_mode'; gave me:
STRICT_TRANS_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION
So I ran this query:
SET GLOBAL sql_mode = 'NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION';
For safety, I will re-enable strict mode after I'm done truncating columns.
I've noticed that, in Laravel, (when using $model->fill(Input::all()), not that it matters how the data comes in), empty fields (empty in a form) come through as an empty string (''). That makes sense, as that's how it's delivered from browser to HTTP server to PHP.
The problem is that if the column is numeric and has a DEFAULT value, or is NULLable, the generated query from Eloquent has '' for the column's value and so MySQL interprets that and enters the value as 0 rather than the default column value or NULL. Is it something I'm doing wrong here, or will I need to put extra work in (e.g. a mutator) to detect this empty string and convert to null to achieve what I actually want?
Of course I understand that from a technical point of view, Laravel, without knowing how your columns work, can't just assume that empty string means pass NULL to the INSERT query, because sometimes you actually want to set a field (specifically a character-based one) to an empty string rather than NULL.
That said, I'd rather not have to define mutators for all my models just because I'm using $model->fill(), but is there anything I don't know about that I can do?
For the MySQL people reading this - is it correct behaviour to set a numeric field to 0 if passed ''? Seems like it should be seen as NULL as it's not explicitly 0, but I guess it's maybe weak typing equating '' to 0 rather than the more distant NULL.
There is a very simple way to do this, and that is by using an array_filter.
$input = array_filter(Input::all(), 'strlen');
$model->fill($input);
The array_filter will return all of the keys that have something assigned to them.
There are some caveats with this solution:
strlen has been used, and not empty. This is because empty will cause other items (such as the number 0) to also be unset.
this means that edits that are made with an update, such as a text box being completely emptied, will not be fulfilled by your application, so use wisely!
EDIT: As for the MySQL question, yes, this is normal.
mysql> SELECT CAST("tim?" AS SIGNED);
+------------------------+
| CAST("tim?" AS SIGNED) |
+------------------------+
| 0 |
+------------------------+
1 row in set, 1 warning (0.00 sec)
mysql> SHOW WARNINGS;
+---------+------+-------------------------------------------+
| Level | Code | Message |
+---------+------+-------------------------------------------+
| Warning | 1292 | Truncated incorrect INTEGER value: 'tim?' |
+---------+------+-------------------------------------------+
For the default values to be inserted on some fields then do not set values for those fields explicitly.
When you receive form data, process each of the fields and decide for which of them you want to insert the default values. Then remove them from insert into ... statement.
Example:
Form fields: field1_1, field_2.
If valid input is given for both of them, then you can write sql statement like:
insert into my_table values( field_1_value, field_2_value );
If one of the fields, say field_2 does not have valid input and you still want the record go into the database table. Then write the sql statement like:
insert into my_table( field_1 ) values( field_1_value );
When you do this, SQL engine will use the default valued defined on the omitted fields in the insert statement.
Note: This only works when you have set default someValue on the columns at the time of creating or altering them in a database table.
I have two tables, pattern and pattern_optimized. Some of the fields in the pattern table are defined as BIGINT(20) DEFAULT '-1', which is way too big for the data they hold; Additionally, -1 is used as "not set" (zero is a valid value) - no other negative values are used otherwise.
The pattern_optimized table uses a new data format for these rows, defining them as INT(10) UNSIGNED DEFAULT NULL. I would now like to copy the data from pattern to the pattern_optimized table.
This would be easily done using INSERT INTO pattern_optimized SELECT * FROM pattern, but obviously all negative values are out of range now, resulting in warnings like this:
100 row(s) affected, 64 warning(s):
1264 Out of range value for column 'version' at row 1
1264 Out of range value for column 'triggered' at row 1
...
Records: 100 Duplicates: 0 Warnings: 357
My first idea was to create a BEFORE INSERT trigger as follows:
CREATE TRIGGER `negativeValueFix` BEFORE INSERT ON `pattern_optimized`
FOR EACH ROW
BEGIN
IF new.version < 0 THEN
SET new.version = NULL;
END IF;
-- ...
END
but unfortunately this doesn't help either: The same warning pops up and all values that used to be -1 in the original table become 0 in the new table (instead of NULL, which is - apart from being implemented in the trigger - also the default for the row).
It seems that MySQL converts the value even before the trigger.
I know that I can solve this problem using a temporary table but I'd rather not do that. The pattern table is unpleasantly large and I don't want to do a stored procedure just for that.
Is there any other way or am I missing some simple point?
EDIT: There are quite alot columns in the original table that suffer from the SIGNED problem, so I was hoping to somewhat automate that.
Could you use a case statement? Something like:
INSERT INTO pattern_optimized
SELECT
CASE version
WHEN -1 THEN null
ELSE version
END CASE AS version
FROM pattern
Just add small condition to the SELECT statement, e.g. -
INSERT INTO pattern_optimized(version)
SELECT IF(version < 0, NULL, version) version FROM pattern
...add other fields.
I have created a table and accidentally put varchar length as 300 instead of 65353. How can I fix that?
An example would be appreciated.
Have you tried this?
ALTER TABLE <table_name> MODIFY <col_name> VARCHAR(65353);
This will change the col_name's type to VARCHAR(65353)
ALTER TABLE <tablename> CHANGE COLUMN <colname> <colname> VARCHAR(65536);
You have to list the column name twice, even if you aren't changing its name.
Note that after you make this change, the data type of the column will be MEDIUMTEXT.
Miky D is correct, the MODIFY command can do this more concisely.
Re the MEDIUMTEXT thing: a MySQL row can be only 65535 bytes (not counting BLOB/TEXT columns). If you try to change a column to be too large, making the total size of the row 65536 or greater, you may get an error. If you try to declare a column of VARCHAR(65536) then it's too large even if it's the only column in that table, so MySQL automatically converts it to a MEDIUMTEXT data type.
mysql> create table foo (str varchar(300));
mysql> alter table foo modify str varchar(65536);
mysql> show create table foo;
CREATE TABLE `foo` (
`str` mediumtext
) ENGINE=MyISAM DEFAULT CHARSET=latin1
1 row in set (0.00 sec)
I misread your original question, you want VARCHAR(65353), which MySQL can do, as long as that column size summed with the other columns in the table doesn't exceed 65535.
mysql> create table foo (str1 varchar(300), str2 varchar(300));
mysql> alter table foo modify str2 varchar(65353);
ERROR 1118 (42000): Row size too large.
The maximum row size for the used table type, not counting BLOBs, is 65535.
You have to change some columns to TEXT or BLOBs
ALTER TABLE {table_name} MODIFY [COLUMN] {column_name} {column_type} {defaults and/or not-null};
(Including COLUMN is optional.)
Note: if your column was created with NOT NULL etc. you may need to specify those in the MODIFY statement to avoid losing them.