I have imported a csv file containing spatial area information in varchar, then converted varchar(max) values to varbinary(max) by adding '0x' to varchar(max) values prior to conversion. By then, apart from the '0x' in the beginning, the data in varbinary(max) column looks exactly the same as the varchar(max) one in converted to text.
Now I run the following script:
select geometry::STGeomCollFromWKB(wkb, 4326) from dbo.MyTable
where WKB is the varbinary(max) column.
Running the above script throws this error: 'The well-known binary (WKB) input is not valid'
The source of data is from Open Street Map so no doubt they are correct area data. So I assume there must be something wrong in what I am doing or I am missing some point to convert WKB to geometry data type.
Could anyone help please?
I assume the problem is when converting the varchar data to varbinary you are converting the actual character representation of the binary data, rather than just changing the type to binary.
Eg, if you have the data 0xDEADBEEF in your varchar column, then doing
convert(varbinary(max), 'DEADBEEF') will convert the ascii character representations into binary.
What you want to do instead is convert the hex string into binary, which is possible using the style parameter of convert.
SELECT convert(varbinary(max), 'DEADBEEF', 2)
should do what you want to convert your varchar wkb data into real binary.
Related
I have a table called messages with a column (BLOB) called message_text. I'm passing a value from the web app when a user sends a new message and I want to check if it's an exact duplicate message text.
SELECT count(message_id) FROM messages WHERE message_text = '$msgTxt' AND user_id = $userId
Where $msgTxt will be a formatted string like...
"Hello there. I don\'t know you.
I\'ve just made a new line. "
The problem is that the comparison isn't working and I'm never finding duplicates. Even if I literally copy/paste an existing value from the database and replace it with $msgTxt in my query I never get any results, and so I'm assuming there's something wrong with the way I'm comparing a blob to a string.
BLOB values are treated as binary strings (byte strings). They have the binary character set and collation, and comparison and sorting are based on the numeric values of the bytes in column values. String or Text values are treated as nonbinary strings (character strings). They have a character set other than binary, and values are sorted and compared based on the collation of the character set.
So, you have to convert either BLOB to String or String to BLOB and then compare both.
If you are using java,
Convert Blob to String
byte[] bdata = blob.getBytes(1, (int)blob.length());
String data1 = new String(bdata);
What API are you using to call MySQL? I see some backslashes, but need to verify that \ is not turning into \\, and that other escapings are not unnecessarily happening or not happening.
Which OS are you using? Windows, when reading stuff, likes to convert NL into CRLF, thereby making it so that it won't match.
I have a column of the image datatype that a piece of software uses to store text for some reason.
I'm trying to pull the data out and convert it to varchar using many examples I've found here.
Currently I have
SELECT CONVERT(VARCHAR(MAX), CONVERT(VARBINARY(MAX), BITS,2)) FROM V_SO_MTL_BINARY
The problem is that only converts the first character from the string for some reason. How do I convert from Image to Varchar and not only get the first character.
You might just need NVARCHAR(MAX) instead of VARCHAR(MAX) depending on how the data is originally inserted.
If it is in fact nvarchar then typical ASCII text will have lots of 00 bytes. e.g. N'Some String' is stored as 0x53006F006D006500200053007400720069006E006700
These will be treated as string terminating null characters when casting back to varchar hence only seeing the first character.
SELECT CAST(0x53006F006D006500200053007400720069006E006700 AS VARCHAR(30))
Returns S
I have MySQL table, and in message field I want to store encrypted data. Encrypted data looks like
�O-�H,,E%P!�O-�H-!E%!P!�O-�H,E%�P!�O-�H,,E$�P"�O-!H,E%P!�O-H+�E%P"
Hence, I cannot store such characters in message either I did utf_general_ci or blog.
Please help me to figure out which datatype can store such characters.
Take a look at this URL: https://dev.mysql.com/doc/refman/5.5/en/encryption-functions.html
"Many encryption and compression functions return strings for which the result might contain arbitrary byte values. If you want to store these results, use a column with a VARBINARY or BLOB binary string data type. This will avoid potential problems with trailing space removal or character set conversion that would change data values, such as may occur if you use a nonbinary string data type (CHAR, VARCHAR, TEXT)."
For archival/historical purposes, after doing an UPDATE on a table in MySQL, I am storing a snapshot of the relevant fields in this table as JSON in a TEXT field of a changelog table. If a table has a varbinary field to securely store sensitive information, I am storing a string representation of the varbinary field by casting the varbinary field to CHAR because I cannot serialize binary data to JSON. This I have done as follows:
SELECT CAST(BinaryField AS CHAR) as CastedValue FROM table
A sample value from the binary field is: 0x774751ECAEC2D03703805E07AB0B8356
and casted value is: wGQ���7�^� �V
The original value was stored in a varbinary field using the MySQL aes_encrypt function and a key.
When I try to decrypt this casted value using aes_decrypt, it is returning NULL:
SELECT cast(aes_decrypt('wGQ���7�^� �V' ,'mykey') as char) as thedata ;
How can I get my original value back from the casted binary value?
Encrypted data is an array of 8-bit bytes with no encoding. In fact many bytes and byte sequences have no UTF-8 (or any meaningful) character representation.
Ultimatly when the data is decrypted it will be back to it's original encoding.
There is no "casting", it is strictly how you look at the bytes, and it is an encoding that gives bytes some meaning other than just 8-bits.
What's difference between BLOB and TEXT data type in mysql ? ( except sortable )
BLOB is used for storing binary data, while TEXT is used to store large strings.
As stated in the MySQL 5.1 Reference Manual:
BLOB values are treated as binary strings (byte strings). They have no
character set, and sorting and comparison are based on the numeric
values of the bytes in column values. TEXT values are treated as
nonbinary strings (character strings). They have a character set, and
values are sorted and compared based on the collation of the character
set.
Mmm google is your friend I guess:
TEXT and CHAR will convert to/from the character set they have associated with time. BLOB and BINARY simply store bytes.
Main difference between BLOB and TEXT: BLOB is casesensetive TEXT.