I was just asking myself if this is standard, because I was setting a column to Type "Char 40" to store a SHA1 value. Is this true? or do I have to pay more attention when I do this in case I work with other then my own mysql database.
Thanks
EDIT
the best possible answer is, that SHA1 just works that way. I thought it was returning 160 bits and some other config setting converted it into a 40 char string, but it always returns that 40 digit string. see doc
SHA1 returns 40 characters, yes.
Related
I am trying to store the number 0.0015 in the database. I have tried float, integer but I am getting zero not the exact figure I have entered. Is there a datatype which can store such a value?
Normally you'd use DECIMAL (aka NUMERIC), with a specified scale and precision, here are the docs for it. FLOAT should also work, but you need to be aware of floating point arithmetics quirks, so DECIMAL is preferred.
Here's a dbfiddle example
If you see your data as 0 then it's either an issue with how you're inserting (maybe importing from a file) or your client rounds it down to 0 and you need to tweak it. As you can see from the dbfiddle above, it works perfectly fine.
This number (0.0015) is not representable in binary. See the following example in python:
Python 3.10.2 (tags/v3.10.2:a58ebcc, Jan 17 2022, 14:12:15) [MSC v.1929 64 bit (AMD64)] on win32
>>> x = 0.0015
>>> (x + 1) - 1
0.0015000000000000568
This means that the storing in mysql (or any other language that converts the number to binary) will show up representation errors. You can use numeric types that doesn't do any conversion to binary, like decimal or numeric.
Bit of an unusual question, but I have setup a field inside a MySQL table that is of the datatype "DECIMAL(5,2)". As far as I understand this, what I have done is to only allow numbers from -999.99 up to 999.99 to be inserted into this field.
However, when I insert an integer to the value of 26 (which is valid) I am shown this inside the database as 26 not 26.00 - is this normal MySQL behaviour? I (perhaps naively) thought that because I have set the scale to 2, my numbers would always be shown with 2 decimal places?
My question is - do integers inside MySQL "DECIMAL" datatypes always display without any decimals places? Or is this my database manager tool formatting 26.00 to 26 for me?
This may seem like a bit of a weird question but I am still trying to get my head around MySQL DECIMALs. Thanks.
I have found the answer out, it seems my MySQL database manager is helpfully (or not so helpfully) hiding the decimals from me. For anyone who finds this vaguely useful I am using EMS SQL Manager 2010 on Windows 7. Credit due to Cylindric who prompted me to check my tools!
In answer to my original question - yes, even integers inside a "DECIMAL" datatype will display the decimal places. For example:
DECIMAL(3,1)
42 = 42.0
531.2 = 99.9
5 = 5.0
Hope this helps someone somewhere!
I'm looking for a shorthand solution to storing an md5 hash inside of a MySQL table, as string data. I had the idea that base256 could reduce the length of the string by half, down to a 16 digit string instead of 32 digits of hex. So I take hex and divide it up into chunks of two digits programatically then convert each set of two digits to ASCII. For example:
4cf5f5941a02573dc007e60442f5358a
is shortened to
Lõõ”W=ÀæBõ5Š
and it's OK if these characters don't print properly - I just need to store them. Would MySQL accept that sort of ASCII data into a text field without complaining?
MySQL will accept these values, but you must be very carefull when writing them - I strongly suggest binding parameters.
You might want to look into COMPRESS() and UNCOMPRESS() as an alternative:
INSERT INTO ... SET hashcode=COMPRESS('4cf5f5941a02573dc007e60442f5358a');
and
SELECT UNCOMPRESS(hashcode) AS hashcode FROM ... WHERE
might do the trick more readable
I am in need to compare data between an Oracle database and a MySQL database.
In Oracle, the data is first encrypted with the AES-128 algorithm, and then hashed. Which means it is not possible to recover the data and decrypt it.
The same data is available in MySQL, and in plain text. So to compare the data, I tried encrypting and then hashing the MySQL data while following the same steps done in Oracle.
After lots of tries, I finally found out that the aes_encrypt in MySQL returns different results than the one in Oracle.
-- ORACLE:
-- First the key is hashed with md5 to make it a 128bit key:
raw_key := DBMS_CRYPTO.Hash (UTL_I18N.STRING_TO_RAW ('test_key', 'AL32UTF8'), DBMS_CRYPTO.HASH_MD5);
-- Initialize the encrypted result
encryption_type:= DBMS_CRYPTO.ENCRYPT_AES128 + DBMS_CRYPTO.CHAIN_CBC + DBMS_CRYPTO.PAD_PKCS5;
-- Then the data is being encrypted with AES:
encrypted_result := DBMS_CRYPTO.ENCRYPT(UTL_I18N.STRING_TO_RAW('test-data', 'AL32UTF8'), encryption_type, raw_key);
The result for the oracle code will be: 8FCA326C25C8908446D28884394F2E22
-- MySQL
-- While doing the same with MySQL, I have tried the following:
SELECT hex(aes_encrypt('test-data', MD5('test_key'));
The result for the MySQL code will be: DC7ACAC07F04BBE0ECEC6B6934CF79FE
Am I missing something? Or are the encryption methods between different languages not the same?
UPDATE:
According to the comments below, I believe I should mention the fact that the result of DBMS_CRYPTO.Hash in Oracle is the same as the result returned by the MD5 function in MySQL.
Also using CBC or CBE in Oracle gives the same result, since the IV isn't being passed to the function, thus the default value of the IV is used which is NULL
BOUNTY:
If someone can verify my last comment, and the fact that if using same padding on both sides, will yield same results gets the bounty:
#rossum The default padding in MySQL is PKCS7, mmm... Oh.. In Oracle
it's using PKCS5, can't believe I didn't notice that. Thanks. (Btw
Oracle doesn't have the PAD_PKCS7 option, not in 11g at least)
MySQL's MD5 function returns a string of 32 hexadecimal characters. It's marked as a binary string but it isn't the 16 byte binary data one would expect.
So to fix it, this string must be converted back to the binary data:
SELECT hex(aes_encrypt('test-data', unhex(MD5('test_key'))));
The result is:
8FCA326C25C8908446D28884394F2E22
It's again a string of 32 hexadecimal characters. But otherwise it's the same result as with Oracle.
And BTW:
MySQL uses PKCS7 padding.
PKCS5 padding and PKCS7 padding are one and the same. So the Oracle padding option is correct.
MySQL uses ECB block cipher mode. So you'll have to adapt the code accordingly. (It doesn't make any difference for the first 16 bytes.)
MySQL uses no initialization vector (the same as your Oracle code).
MySQL uses a non-standard folding a keys. So to achieve the same result in MySQL and Oracle (or .NET or Java), only use keys that are 16 byte long.
Just would like to give the complete solution for dummies based on #Codo's very didactic answer.
EDIT:
For being exact in general cases, I found this:
- "PKCS#5 padding is a subset of PKCS#7 padding for 8 byte block sizes".
So strictly PKCS5 can't be applied to AES; they mean PKCS7 but use their
names interchangeably.
About PKCS5 and PKCS7
/* MySQL uses a non-standard folding a key.
* So to achieve the same result in MySQL and Oracle (or .NET or Java),
only use keys that are 16 bytes long (32 hexadecimal symbols) = 128 bits
AES encryption, the MySQL AES_encrypt default one.
*
* This means MySQL admits any key length between 16 and 32 bytes
for 128 bits AES encryption, but it's not allowed by the standard
AES to use a non-16 bytes key, so do not use it as you won't be able
to use the standard AES decrypt in other platform for keys with more
than 16 bytes, and would be obligued to program the MySQL folding of
the key in that other platform, with the XOR stuff, etc.
(it's already out there but why doing weird non-standard things thay
may change when MySQL decide, etc.).
Moreover, I think they say the algorithm chosen by MySQL for those
cases is a really bad choose on a security level...
*/
-- ### ORACLE:
-- First the key is hashed with md5 to make it a 128 bit key (16 bytes, 32 hex symbols):
raw_key := DBMS_CRYPTO.Hash (UTL_I18N.STRING_TO_RAW ('test_key', 'AL32UTF8'), DBMS_CRYPTO.HASH_MD5);
-- MySQL uses AL32UTF8, at least by default
-- Configure the encryption parameters:
encryption_type:= DBMS_CRYPTO.ENCRYPT_AES128 + DBMS_CRYPTO.CHAIN_ECB + DBMS_CRYPTO.PAD_PKCS5;
-- Strictly speaking, it's really PKCS7.
/* And I choose ECB for being faster if applied and
#Codo said it's the correct one, but as standard (Oracle) AES128 will only accept
16 bytes keys, CBC also works, as I believe they are not applied to a 16 bytes key.
Could someone confirm this? */
-- Then the data is encrypted with AES:
encrypted_result := DBMS_CRYPTO.ENCRYPT(UTL_I18N.STRING_TO_RAW('test-data', 'AL32UTF8'), encryption_type, raw_key);
-- The result is binary (varbinary, blob).
-- One can use RAWTOHEX() for if you want to represent it in hex characters.
In case you use directly the 16 bytes hashed passphrase in hex characters representation or 32 hex random chars:
raw_key := HEXTORAW(32_hex_key)
encryption_type := 6 + 768 + 4096 -- (same as above in numbers; see Oracle Docum.)
raw_data := UTL_I18N.STRING_TO_RAW('test-data', 'AL32UTF8')
encrypted_result := DBMS_CRYPTO.ENCRYPT( raw_data, encryption_type, raw_key )
-- ORACLE Decryption:
decrypted_result := UTL_I18N.RAW_TO_CHAR( CRYPTO.DECRYPT( raw_data, encryption_type, raw_key ), 'AL32UTF8' )
-- In SQL:
SELECT
UTL_I18N.RAW_TO_CHAR(
DBMS_CRYPTO.DECRYPT(
UTL_I18N.STRING_TO_RAW('test-data', 'AL32UTF8'),
6 + 768 + 4096,
HEXTORAW(32_hex_key)
) , 'AL32UTF8') as "decrypted"
FROM DUAL;
-- ### MySQL decryption:
-- MySQL's MD5 function returns a string of 32 hexadecimal characters (=16 bytes=128 bits).
-- It's marked as a binary string but it isn't the 16 bytes binary data one would expect.
-- NOTE: Note that the kind of return of MD5, SHA1, etc functions changed in some versions since 5.3.x. See MySQL 5.7 manual.
-- So to fix it, this string must be converted back from hex to binary data with unHex():
SELECT hex(aes_encrypt('test-data', unhex(MD5('test_key')));
P.S.:
I would recommend to read the improved explanation in MySQL 5.7 Manual, which moreover now allows a lot more configuration.
MySQL AES_ENCRYPT improved explanation from v5.7 manual
Could be CBC vs ECB. Comment at the bottom of this page: http://dev.mysql.com/doc/refman/5.5/en/encryption-functions.html says mysql function uses ECB
I want to store a hashed password (using BCrypt) in a database. What would be a good type for this, and which would be the correct length? Are passwords hashed with BCrypt always of same length?
EDIT
Example hash:
$2a$10$KssILxWNR6k62B7yiX0GAe2Q7wwHlrzhF3LqtVvpyvHZf0MwvNfVu
After hashing some passwords, it seems that BCrypt always generates 60 character hashes.
EDIT 2
Sorry for not mentioning the implementation. I am using jBCrypt.
The modular crypt format for bcrypt consists of
$2$, $2a$ or $2y$ identifying the hashing algorithm and format
a two digit value denoting the cost parameter, followed by $
a 53 characters long base-64-encoded value (they use the alphabet ., /, 0–9, A–Z, a–z that is different to the standard Base 64 Encoding alphabet) consisting of:
22 characters of salt (effectively only 128 bits of the 132 decoded bits)
31 characters of encrypted output (effectively only 184 bits of the 186 decoded bits)
Thus the total length is 59 or 60 bytes respectively.
As you use the 2a format, you’ll need 60 bytes. And thus for MySQL I’ll recommend to use the CHAR(60) BINARYor BINARY(60) (see The _bin and binary Collations for information about the difference).
CHAR is not binary safe and equality does not depend solely on the byte value but on the actual collation; in the worst case A is treated as equal to a. See The _bin and binary Collations for more information.
A Bcrypt hash can be stored in a BINARY(40) column.
BINARY(60), as the other answers suggest, is the easiest and most natural choice, but if you want to maximize storage efficiency, you can save 20 bytes by losslessly deconstructing the hash. I've documented this more thoroughly on GitHub: https://github.com/ademarre/binary-mcf
Bcrypt hashes follow a structure referred to as modular crypt format (MCF). Binary MCF (BMCF) decodes these textual hash representations to a more compact binary structure. In the case of Bcrypt, the resulting binary hash is 40 bytes.
Gumbo did a nice job of explaining the four components of a Bcrypt MCF hash:
$<id>$<cost>$<salt><digest>
Decoding to BMCF goes like this:
$<id>$ can be represented in 3 bits.
<cost>$, 04-31, can be represented in 5 bits. Put these together for 1 byte.
The 22-character salt is a (non-standard) base-64 representation of 128 bits. Base-64 decoding yields 16 bytes.
The 31-character hash digest can be base-64 decoded to 23 bytes.
Put it all together for 40 bytes: 1 + 16 + 23
You can read more at the link above, or examine my PHP implementation, also on GitHub.
If you are using PHP's password_hash() with the PASSWORD_DEFAULT algorithm to generate the bcrypt hash (which I would assume is a large percentage of people reading this question) be sure to keep in mind that in the future password_hash() might use a different algorithm as the default and this could therefore affect the length of the hash (but it may not necessarily be longer).
From the manual page:
Note that this constant is designed to change over time as new and
stronger algorithms are added to PHP. For that reason, the length of
the result from using this identifier can change over time. Therefore,
it is recommended to store the result in a database column that can
expand beyond 60 characters (255 characters would be a good choice).
Using bcrypt, even if you have 1 billion users (i.e. you're currently competing with facebook) to store 255 byte password hashes it would only ~255 GB of data - about the size of a smallish SSD hard drive. It is extremely unlikely that storing the password hash is going to be the bottleneck in your application. However in the off chance that storage space really is an issue for some reason, you can use PASSWORD_BCRYPT to force password_hash() to use bcrypt, even if that's not the default. Just be sure to stay informed about any vulnerabilities found in bcrypt and review the release notes every time a new PHP version is released. If the default algorithm is ever changed it would be good to review why and make an informed decision whether to use the new algorithm or not.
I don't think that there are any neat tricks you can do storing this as you can do for example with an MD5 hash.
I think your best bet is to store it as a CHAR(60) as it is always 60 chars long
I think best choice is nonbinary type, because in comparison is less combination and should be faster. If data is encoded with base64_encode then each position, each byte have only 64 possible values. If encoded with bin2hex each byte have only 16 possible values, but string is much longer. In binary byte have 256 position on each.
I use for hashes in form of encode64 VARCHAR(255) column with ascii character set and the same collation.
VARBINARY causes comparison problem as described in MySQL documentation. I don't know why answers advice to use VARBINARY have so many positives.
I checked this on my author site, where measure time (just refresh to see).