For archival/historical purposes, after doing an UPDATE on a table in MySQL, I am storing a snapshot of the relevant fields in this table as JSON in a TEXT field of a changelog table. If a table has a varbinary field to securely store sensitive information, I am storing a string representation of the varbinary field by casting the varbinary field to CHAR because I cannot serialize binary data to JSON. This I have done as follows:
SELECT CAST(BinaryField AS CHAR) as CastedValue FROM table
A sample value from the binary field is: 0x774751ECAEC2D03703805E07AB0B8356
and casted value is: wGQ���7�^� �V
The original value was stored in a varbinary field using the MySQL aes_encrypt function and a key.
When I try to decrypt this casted value using aes_decrypt, it is returning NULL:
SELECT cast(aes_decrypt('wGQ���7�^� �V' ,'mykey') as char) as thedata ;
How can I get my original value back from the casted binary value?
Encrypted data is an array of 8-bit bytes with no encoding. In fact many bytes and byte sequences have no UTF-8 (or any meaningful) character representation.
Ultimatly when the data is decrypted it will be back to it's original encoding.
There is no "casting", it is strictly how you look at the bytes, and it is an encoding that gives bytes some meaning other than just 8-bits.
Related
The Mysql db I am using has char encoding utf8 but certain sets of chars which take beyond 3 bytes are not getting saved. I could have changed the encoding to utf8mb4 but that is not an option. All I want to do is validate a string to check if the string will get saved in Mysql. I don't want to unnecessarily limit my chars to ASCII. How do I check if a char will take more than three bytes?
Plan A:
In your app language, convert your string to hex. Then look for f0. That byte would indicate utf8mb4 is needed.
In MySQL, the expression is HEX(col) REGEXP '^(..)*f0'.
Plan B:
Attempt to insert your text into a CHARACTER SET utf8 column of a spare table. Read it back and see if it matches. Storing a 4-byte character will either turn it into question marks or truncate the string. Either way, it won't match.
If you wish to insert the data in mysql query only rather than programmaticaly, then you can use the length() function to check for the byte length.
MySQL provides the LENGTH function to get a length of a string in bytes, and the CHAR_LENGTH function to get the length of a string in characters. If a string contains the multi-bytes character, the result of the LENGTH function is greater than the result of the CHAR_LENGTH() function
http://www.mysqltutorial.org/mysql-character-set/
Sample query follows
insert into x_table(data_string)
SELECT 'šč' as data_string where length('šč')<4
in Java check length before inserting into mysql
using
String s = new String("stringvalue");
byte[] bytes = s.getBytes("UTF-8");
System.out.println("bytes.length = "+bytes.length);
bytes.length can be checked before inserting (String.getBytes().length).
I have a table called messages with a column (BLOB) called message_text. I'm passing a value from the web app when a user sends a new message and I want to check if it's an exact duplicate message text.
SELECT count(message_id) FROM messages WHERE message_text = '$msgTxt' AND user_id = $userId
Where $msgTxt will be a formatted string like...
"Hello there. I don\'t know you.
I\'ve just made a new line. "
The problem is that the comparison isn't working and I'm never finding duplicates. Even if I literally copy/paste an existing value from the database and replace it with $msgTxt in my query I never get any results, and so I'm assuming there's something wrong with the way I'm comparing a blob to a string.
BLOB values are treated as binary strings (byte strings). They have the binary character set and collation, and comparison and sorting are based on the numeric values of the bytes in column values. String or Text values are treated as nonbinary strings (character strings). They have a character set other than binary, and values are sorted and compared based on the collation of the character set.
So, you have to convert either BLOB to String or String to BLOB and then compare both.
If you are using java,
Convert Blob to String
byte[] bdata = blob.getBytes(1, (int)blob.length());
String data1 = new String(bdata);
What API are you using to call MySQL? I see some backslashes, but need to verify that \ is not turning into \\, and that other escapings are not unnecessarily happening or not happening.
Which OS are you using? Windows, when reading stuff, likes to convert NL into CRLF, thereby making it so that it won't match.
I have MySQL table, and in message field I want to store encrypted data. Encrypted data looks like
�O-�H,,E%P!�O-�H-!E%!P!�O-�H,E%�P!�O-�H,,E$�P"�O-!H,E%P!�O-H+�E%P"
Hence, I cannot store such characters in message either I did utf_general_ci or blog.
Please help me to figure out which datatype can store such characters.
Take a look at this URL: https://dev.mysql.com/doc/refman/5.5/en/encryption-functions.html
"Many encryption and compression functions return strings for which the result might contain arbitrary byte values. If you want to store these results, use a column with a VARBINARY or BLOB binary string data type. This will avoid potential problems with trailing space removal or character set conversion that would change data values, such as may occur if you use a nonbinary string data type (CHAR, VARCHAR, TEXT)."
I have imported a csv file containing spatial area information in varchar, then converted varchar(max) values to varbinary(max) by adding '0x' to varchar(max) values prior to conversion. By then, apart from the '0x' in the beginning, the data in varbinary(max) column looks exactly the same as the varchar(max) one in converted to text.
Now I run the following script:
select geometry::STGeomCollFromWKB(wkb, 4326) from dbo.MyTable
where WKB is the varbinary(max) column.
Running the above script throws this error: 'The well-known binary (WKB) input is not valid'
The source of data is from Open Street Map so no doubt they are correct area data. So I assume there must be something wrong in what I am doing or I am missing some point to convert WKB to geometry data type.
Could anyone help please?
I assume the problem is when converting the varchar data to varbinary you are converting the actual character representation of the binary data, rather than just changing the type to binary.
Eg, if you have the data 0xDEADBEEF in your varchar column, then doing
convert(varbinary(max), 'DEADBEEF') will convert the ascii character representations into binary.
What you want to do instead is convert the hex string into binary, which is possible using the style parameter of convert.
SELECT convert(varbinary(max), 'DEADBEEF', 2)
should do what you want to convert your varchar wkb data into real binary.
What's difference between BLOB and TEXT data type in mysql ? ( except sortable )
BLOB is used for storing binary data, while TEXT is used to store large strings.
As stated in the MySQL 5.1 Reference Manual:
BLOB values are treated as binary strings (byte strings). They have no
character set, and sorting and comparison are based on the numeric
values of the bytes in column values. TEXT values are treated as
nonbinary strings (character strings). They have a character set, and
values are sorted and compared based on the collation of the character
set.
Mmm google is your friend I guess:
TEXT and CHAR will convert to/from the character set they have associated with time. BLOB and BINARY simply store bytes.
Main difference between BLOB and TEXT: BLOB is casesensetive TEXT.