I keep finding that MySQL Workbench shows query results as BLOB. e.g:
SELECT INET_NTOA(167773449) --> BLOB
If I select to 'view value' I can determine the text value is '10.0.5.9' but it's pretty irritating when I SELECT multiple rows and want to glance at the contents.
Is there a way around this or is it a limitation of the tool?
Background:
This problem occurs when the binary string values (BINARY/VARBINARY type) are returned in the results. The binary strings contain the zero bytes and for some reason, apparently security, have not been shown by default. More details about binary strings here.
Even in the reported example SELECT INET_NTOA(167773449), the function returns binary string. Check this for reference.
Solution:
Since MySQL Workbench v5.2.22, it can be set through preferences whether to SHOW or HIDE such values.
In MySQL Workbench, go to: "Edit -> Preferences... -> SQL Queries" OR "Edit -> Preferences... -> SQL Editor -> SQL Execution" (depending upon what version of Workbench you have).
Check the option 'Treat BINARY/VARBINARY as nonbinary character string' to show the actual value.
Reference:
The original issue has been reported and answered with fix here.
What you can do is Cast your BLOB type to a string. This will simply allow you to glance at whats in your BLOB type when browsing your select statement.
SELECT CAST(`blob_column_name` AS CHAR(10000) CHARACTER SET utf8) FROM `table_name`;
Another (slightly shorter) solution
SELECT CONVERT(FROM_BASE64(myMEDIUMTEXTcol) using utf8) notAblobject FROM myTable;
Unrelated background
My table contains a MEDIUMTEXT column which stores a Base64 string. In my frontend JS code I kept getting an Object returned instead of a string. The Object appeared to contain an array of ASCII values.
Testing the associated query in phpMyAdmin indicated the result was a BLOB. I updated my SQL query, as above, in order to get a simple string returned instead of an Object.
Related
I have a MySQL table with a VARCHAR(100) column, using the utf8_general_ci collation.
I can see rows where this column contains arbitrary byte sequences (i.e. data that contains invalid UTF8 character sequences), but I can't figure out how to write an UPDATE or INSERT statement that allows this type of data to be entered.
For example, I've tried the following:
UPDATE DataTable SET Data = CAST(BINARY(X'16d7a4fca7442dda3ad93c9a726597e4') AS CHAR(100)) WHERE Id = 1;
But I get the error:
Incorrect string value: '\xFC\xA7D-\xDA:...' for column 'Data' at row 1
How can I write an INSERT or UPDATE statement that bypasses the destination column's collation, allowing me to insert arbitrary byte sequences?
Have you considered using one of the Blob data types instead of varchar? I believe that this'd take a lot of the pain away from your use-case.
EDIT: Alternatively, there is the HEX and UNHEX functions, which MySQL supports. Hex takes either a str or a numeric argument and returns the hexadecimal representation of your argument as a string. Unhex does the inverse; taking a hexadecimal string and returning a binary string.
The short answer is that it shouldn't be possible to insert values with invalid UTF8 characters into VARCHAR column declared to use UTF8 characterset.
That's the design goal of MySQL, to disallow invalid values. When there's an attempt to do that, MySQL will return either an error or a warning, or (more leniently?) silently truncate the supplied value at the first invalid character encountered.
The more usual variety of characterset issues are with MySQL performing a characterset conversion when a characterset conversion isn't required.
But the issue you are reporting is that invalid characters were inserted into a UTF8 column. It's as if a latin1 (ISO-8859) encoding was supplied, and a characterset conversion was required, but was not performed.
As far as working around that... I believe it was possible in earlier versions of MySQL. I believe it was possible to cast a value to BINARY, and then warp that in CONVERT( ... USING UTF8), and MySQL wouldn't perform a validation of the characterset. I don't know if that's still possible with the current MySQL Connectors.
If it is possible, then that's (IMO) a bug in the Connector.
The only way I can think of getting around that characterset check/validation would be to get the MySQL sever to trust the client, and determine that no check of the characterset is required. (That would also mean the MySQL server wouldn't be doing a characterset conversion, the client lying to the server, the client telling the server that it's supplying valid UTF8 characters.
Basically, the client would be telling the server "Hey server, I'm going to be sending UTF8 character encodings".
And the server says "Okay. I'll not do any characterset conversion then, since we match. And I'll just trust that what you send is valid UTF8".
And then the client mischievously chuckles to itself, "Heh, heh, I lied. I'm actually sending character encodings that aren't valid UTF8".
And I think it's much more likely to be able to achieve such mischief using prepared statements with the old school MySQL C API (mysql_stmt_prepare, mysql_stmt_execute), supplying nvalid UTF8 encodings as values for string bind parameters. (The onus is really on the client to supply valid values for bind parameters.)
You should base64 encode your value beforehand so you can generate a valid SQL with it:
UPDATE DataTable SET Data = from_base64('mybase64-encoded-representation-of-my-value') WHERE Id = 1;
In the process of moving some applications from PHP and Delphi to Java-RS some column entries are spoiling the show at this time.
After reading the data from mySQL with JPA into Java Pojos converting the result to XML using JaxB and trying to read back the result to JQuery / jqGrid a failure happens.
In the browser the problem simply shows at "not well formed".
Looking at the details with the Eclipse XML editor gives the error message:
An invalid XML character (Unicode 0x0) was found in the element content of the document
Now I'd like to proceeed and fix the original data.
How would an SQL query look like that looks for the rows that have
invalid entries?
How would an SQL query look like that fixes these
rows?
Let's assume the column with the problem is "name" in Table "Customer"
For question #1 I found:
SELECT name from customer where hex(name) like "%00";
to work. For question #2 I am assuming that update with a left substring might work. I am not sure about the length in this case. It looks like length(name) will return the length including the terminating zero character. Will the update with left(length(name)-1) work correctly in this case?
I am not sure whether backup and restore of the database would keep the current somewhat corrupted database in shape. So it would also be helpful to know how the situation can be reproduced with an insert statement that creates null terminated strings on purpose.
I think you should be able to transform the HEX() with something like
UPDATE customer SET name=UNHEX(REPLACE(HEX(name), '00', ''));
or simply
UPDATE customer SET name=REPLACE(name, CHAR(0), '');
update customer set name=left(name,length(name)-1) where hex(name) like "%00";
I am trying to return an MD5 encoded string of a value from my database, but it just returns a blank result (not null, just blank).
I have tried just running this query and get the same result:
SELECT MD5('test');
I have tried restarting the MySQL server, MySQL Workbench, etc. But get the same result.
If I try running the same command on a different database/server, it returns the hash string just fine.
What am I doing wrong? Is there a setting I disabled on accident?
Prior to MySQL v5.5.3, MD5() returned a binary string.
By default, MySQL Workbench does not display binary strings (to avoid accidental misinterpretation); however it is possible to display binary string values in output grids: View > Edit > Preferences > SQL Editor > Treat BINARY/VARBINARY as nonbinary character string.
Alternatively, either upgrade your MySQL server or transcode the result to a non-binary character set:
SELECT CONVERT(MD5('test') USING utf8)
I have a sql table that gets populated via SQLBulkCopy from Excel. The copy down is done using the Microsoft ACE drivers.
I had a problem with one particular file - when it was loaded down to sql, some of the columns (which appear empty in excel) contained an odd value.
For example, running this sql:
SELECT
CONVERT(VARBINARY(10),MyCol),
LEN(MyCol)
FROM MyTab
would return
0x, 0
i.e. - converting the value in the column to varbinary shows something, but doing length of the varchar shows no length. I realise that the value shown is the stem of a hex value, but its weird that its gets there, and how hard it is to detect.
Obviously I can just clear out the cells in Excel, but I really need to detect this automatically as end users will have the same issue. It is causing issues further down the line when the data gets processed. Its quite hard to trace the problem back from its eventual symptoms to being this issue in the source.
Other than the above conversion to varbinary to output in SSMS I've not come up with a way of detecting these values, either in Excel or via a SQL script to remove them.
Any ideas?
This may help you:
-- Conversion from hex string to varbinary:
DECLARE #hexstring VarChar(MAX);
SET #hexstring = 'abcedf012439';
SELECT CAST('' AS XML).Value('xs:hexBinary( substring(sql:variable("#hexstring"), sql:column("t.pos")) )', 'varbinary(max)')
FROM (SELECT CASE SubString(#hexstring, 1, 2) WHEN '0x' THEN 3 ELSE 0 END) AS t(pos)
GO
-- Conversion from varbinary to hex string:
DECLARE #hexbin VarBinary(MAX);
SET #hexbin = 0xabcedf012439;
SELECT '0x' + CAST('' AS XML).Value('xs:hexBinary(sql:variable("#hexbin") )', 'varchar(max)');
GO
One method is to add a new column, convert the data, drop the
old column and rename the new column to the old name.
As Martin points out above, 0x is what you get when you convert an empty string. eg:
SELECT CONVERT(VARBINARY(10),'')
So the problem of detecting it obviously goes away.
I have to assume that there is some rubbish in the excel cell, that is being filtered out in the process of the write down by either the ACE driver or the SQLBulkCopy. Because there was something in the field originally, the value written is empty instead of null.
In order to make sure that everything is consistent in the data we'll need to do a post process to switch all empty values to nulls so that the next lots of scripts work.
I'm using MySQL Workbench CE 5.2.30 CE / Rev 6790 . When execute the following statement:
SELECT OLD_PASSWORD("test")
I only get back a nice BLOB icon, I need to left-click to select the cell, right-click and choose "Open Value in viewer" and select the "Text" tab.
Using the same with phpMyAdmin, I get directly back the value of the OLD_PASSWORD call. It's just an example, but is there a way to directly see such results in the output?
In short:
Go to Edit > Preferences
Choose SQL Editor
Under SQL Execution, check Treat BINARY/VARBINARY as nonbinary character string
Restart MySQL Workbench (you will not be prompted or informed of this requirement).
In MySQL Workbench 6.0+
Go to Edit > Preferences
Choose SQL Queries
Under Query Results, check Treat BINARY/VARBINARY as nonbinary character string
It's not mandatory to restart MySQL Workbench (you will not be prompted or informed of this requirement).*
With this setting you will be able to concatenate fields without getting blobs.
I think this applies to versions 5.2.22 and later and is the result of this MySQL bug.
Disclaimer: I don't know what the downside of this setting is - maybe when you are selecting BINARY/VARBINARY values you will see it as plain text which may be misleading and/or maybe it will hinder performance if they are large enough?
I'm not sure if this answers the question but if if you right click on the "blob" icon in the field (when viewing the table) there is an option to "Open Value in Editor". One of the tabs lets you view the blob. This is in ver. 5.2.34
Perform three steps:
Go to "WorkBench Preferences" --> Choose "SQL Editor" Under "Query Results": check "Treat BINARY/VARBINARY as nonbinary character string"
Restart MySQL WorkBench.
Now select SELECT SUBSTRING(<BLOB_COLUMN_NAME>,1,2500) FROM <Table_name>;
casting works, but it is a pain, so I would recommend using spioter's method unless you are using a lot of truly blob data.
SELECT CAST(OLD_PASSWORD("test") AS CHAR)
You can also cast as other types, and even restrict the size, but most of the time I just use CHAR:
http://dev.mysql.com/doc/refman/5.5/en/cast-functions.html#function_cast
select CONVERT((column_name) USING utf8) FROM table;
In my case, Workbench does not work. so i used the above solution to show blob data as text.
Doesn't seem to be possible I'm afraid, its listed as a bug in workbench:
http://bugs.mysql.com/bug.php?id=50692
It would be very useful though!
had the same problem, according to the MySQL documentation, you can select a Substring of a BLOB:
SELECT id, SUBSTRING(comment,1,2000) FROM t
HTH, glissi
I pieced a few of the other posts together, as the workbench 'preferences' fix did not work for me. (WB 6.3)
SELECT CAST(`column` AS CHAR(10000) CHARACTER SET utf8) FROM `table`;
Work bench 6.3
Follow High scoring answer then use UNCOMPRESS()
(In short:
1. Go to Edit > Preferences
2. Choose SQL Editor
3. Under SQL Execution, check Treat BINARY/VARBINARY as nonbinary character string
4. Restart MySQL Workbench (you will not be prompted or informed of this requirement).)
Then
SELECT SUBSTRING(UNCOMPRESS(<COLUMN_NAME>),1,2500) FROM <Table_name>;
or
SELECT CAST(UNCOMPRESS(<COLUMN_NAME>) AS CHAR) FROM <Table_name>;
If you just put UNCOMPRESS(<COLUMN_NAME>) you can right click blob and click "Open Value in Editor".
there is few things that you can do
SELECT GROUP_CONCAT(CAST(name AS CHAR))
FROM product
WHERE id IN (12345,12346,12347)
If you want to order by the query you can order by cast as well like below
SELECT GROUP_CONCAT(name ORDER BY name))
FROM product
WHERE id IN (12345,12346,12347)
as it says on this blog
http://www.kdecom.com/mysql-group-concat-blob-bug-solved/
NOTE: The previous answers here aren't particularly useful if the BLOB is an arbitrary sequence of bytes; e.g. BINARY(16) to store 128-bit GUID or md5 checksum.
In that case, there currently is no editor preference -- though I have submitted a feature request now -- see that request for more detailed explanation.
[Until/unless that feature request is implemented], the solution is HEX function in a query: SELECT HEX(mybinarycolumn) FROM mytable.
An alternative is to use phpMyAdmin instead of MySQL Workbench - there hex is shown by default.