I am trying to get rid of   characters in MYSQL, but am getting weird behavior where using REPLACE is returning a hexadecimal string.
The original value is some HTML stored in a field with the type BLOB:
<h3>This was just an appetizer. Are you ready for the full course?</h3><p>Dive into more business news, check out the latest tech trends, and get a couple quick tips from our health section. </p></div>
The SQL I am using is this:
UPDATE tbl
SET field = REPLACE(field, CHAR(160), '');
And after executing, this is what is left in the database:
3C68333E5468697320776173206A75737420616E206170706574697A65722E2041726520796F7520726561647920666F72207468652066756C6C20636F757273653F3C2F68333E3C703E4469766520696E746F206D6F726520627573696E657373206E6577732C20636865636B206F757420746865206C61746573742074656368207472656E64732C20616E6420676574206120636F75706C6520717569636B20746970732066726F6D206F7572206865616C74682073656374696F6E2E20C23C2F703E3C2F6469763E
What is going on and how could I avoid this? Do I need to use VARCHAR for the field type?
You get (binary) BLOB back, after the replace.
so you have to convert it back to text
UPDATE tbl
SET field = CAST(REPLACE(field, CHAR(160), '')AS CHAR(10000) CHARACTER SET utf8);
Of course you have to check character set and size.
I found that CHAR codes didn't work, but a copy pasted whitespace worked. This looks like a normal space, but is in fact CHAR(160) and I don't have an error anymore. ' '
Using MySQLAdmin. Moved data from Windows server and trying to replace case in urls but not finding the matches. Need slashes as I don't want to replace text in anything but the urls (in post table). I think the %20 are the problem somwhow?
UPDATE table_name SET field = replace(field, '/user%20name/', '/User%20Name/')
The actual string is more like:
https://www.example.com/forum/uploads/user%20name/GFCI%20Stds%20Rev%202006%20.pdf
In a case you are using MariaDB you have REGEXP_REPLACE() function.
But best approach is to dump the table into the file. Open it in a Notepad ++
and run regex replace like specified on a pic:
Pattern is: (https:[\/\w\s\.]+uploads/)(\w+)\%20(\w+)((\/.*)+)
Replace with: $1\u$2\%20\u$3$4
Then import the table again
Hope this help
If its MariaDB, you can do the following:
UPDATE table_name SET field = REGEXP_REPLACE(field, '\/user%20name\/', '\/User%20Name\/');
First, please check, what is actually stored in the database: %20 is a html-entity which represents a whitespace. Usually, when you are storing this inside the database, it will be represented as an actual whitespace (converted before you store it) -> Hence your replace doesn't match the actual data.
The second option that might be possible - depending on what you want to do: You are seeing the URL containing %20, therefore you created your database records (which you would like to fetch) with that additional %20 - And when you now try to query your results based on the actual url, the %20 is replaced with an "actual" whitespace (before your query) and hence it doesn't match your stored data.
I've written a MySQL script to create a database for hypothetical hospital records and populate it with data. One of the tables, Department, has a column named Description, which is declared as type varchar(200). When executing the INSERT command for Description I get an error:
error 1406: Data too long for column 'Description' at row 1.
All the strings I'm inserting are less than 150 characters.
Here's the declaration:
CREATE TABLE Department(
...
Description varchar(200)
...);
And here's the insertion command:
INSERT INTO Department VALUES
(..., 'There is some text here',...), (..., 'There is some more text over here',...);
By all appearances, this should be working. Anyone have some insight?
Change column type to LONGTEXT
I had a similar problem when migrating an old database to a new version.
Switch the MySQL mode to not use STRICT.
SET ##global.sql_mode= 'NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION';
Error Code: 1406. Data too long for column - MySQL
There is an hard limit on how much data can be stored in a single row of a mysql table, regardless of the number of columns or the individual column length.
As stated in the OFFICIAL DOCUMENTATION
The maximum row size constrains the number (and possibly size) of columns because the total length of all columns cannot exceed this size. For example, utf8 characters require up to three bytes per character, so for a CHAR(255) CHARACTER SET utf8 column, the server must allocate 255 × 3 = 765 bytes per value. Consequently, a table cannot contain more than 65,535 / 765 = 85 such columns.
Storage for variable-length columns includes length bytes, which are assessed against the row size. For example, a VARCHAR(255) CHARACTER SET utf8 column takes two bytes to store the length of the value, so each value can take up to 767 bytes.
Here you can find INNODB TABLES LIMITATIONS
in mysql if you take VARCHAR then change it to TEXT bcoz its size is 65,535
and if you can already take TEXT the change it with LONGTEXT only if u need more then 65,535.
total size of LONGTEXT is 4,294,967,295 characters
Varchar has its own limits. Maybe try changing datatype to text.!
Turns out, as is often the case, it was a stupid error on my part. The way I was testing this, I wasn't rebuilding the Department table after changing the data type from varchar(50) to varchar(200); I was just re-running the insert command, still with the column as varchar(50).
If your source data is larger than your target field and you just want to cut off any extra characters, but you don't want to turn off strict mode or change the target field's size, then just cut the data down to the size you need with LEFT(field_name,size).
INSERT INTO Department VALUES
(..., LEFT('There is some text here',30),...), (..., LEFT('There is some more text over here',30),...);
I used "30" as an example of your target field's size.
In some of my code, it's easy to get the target field's size and do this. But if your code makes that hard, then go with one of the other answers.
For me, I defined column type as BIT (e.g. "boolean")
When I tried to set column value "1" via UI (Workbench), I was getting a "Data too long for column" error.
Turns out that there is a special syntax for setting BIT values, which is:
b'1'
With Hibernate you can create your own UserType. So thats what I did for this issue. Something as simple as this:
public class BytesType implements org.hibernate.usertype.UserType {
private final int[] SQL_TYPES = new int[] { java.sql.Types.VARBINARY };
//...
}
There of course is more to implement from extending your own UserType but I just wanted to throw that out there for anyone looking for other methods.
Very old question, but I tried everything suggested above and still could not get it resolved.
Turns out that, I had after insert/update trigger for the main table which tracked the changes by inserting the record in history table having similar structure. I increased the size in the main table column but forgot to change the size of history table column and that created the problem.
I did similar changes in the other table and error is gone.
I try to create a table with a field as 200 characters and I've added two rows with early 160 characters and it's OK. Are you sure your rows are less than 200 characters?
Show SqlFiddle
There was a similar problem when storing a hashed password into a table. Changing the maximum column length didn't help. Everything turned out to be simple. It was necessary to delete the previously created table from the database, and then test the code with new values of the allowable length.
If you re using type: DataTypes.STRING, then just pass how long this string can be like DataTypes.STRING(1000)
In my case this error occurred due to entering data a wrong type for example: if it is a long type column, i tried to enter in string type. so please check your data that you are entering and type are same or not
For me, I try to update column type "boolean" value
When I tried to set column value 1 MySQL Workbench, I was getting a "Data too long for column" error.
So for that there is a special syntax for setting boolean values, which is:
UPDATE `DBNAME`.`TABLE_NAME` SET `FIELD_NAME` = false WHERE (`ID` = 'ID_VALUE'); //false for 0
UPDATE `DBNAME`.`TABLE_NAME` SET `FIELD_NAME` = true WHERE (`ID` = 'ID_VALUE'); //true for 1
I had a different problem which gave the same error so I'll make a quick recap as this seems to have quite different sources and the error does not help much to track down the root cause.
Common sources for INSERT / UPDATE
Size of value in row
This is exactly what the error is complaining about. Maybe it's just that.
You can:
increase the column size: for long strings you can try to use TEXT, MEDIUMTEXT or LONGTEXT
trim the value that is too long: you can use tools from the language you're using to build the query or directly in SQL with LEFT(value,size) or RIGHT(...) or SUBSTRING(...)
Beware that there is a maximum row size in a MySQL table as reported by this answer. Check documentation and InnoDB engine limitations.
Datatype Mismatch
One or more rows are of the wrong datatype.
common sources of error are
ENUM
BIT: don't use 1 but b'1'
Data outlier
In a long list of insert one can easily miss a row which has a field not adhering to the column typing, like an ENUM generated from a string.
Python Django
Check if you have sample_history enabled, after a change in a column size it must be updated too.
I've got a huge mysql table (called tcountriesnew) and a column (called slogen, blob type).
In each of those slogen blobs I'd like to replace a word, for example: banana to apple.
Unfortunately I tried to print all the rows with word banana, and it did not work.
select * from tcountriesnew where slogen like '%banana%';
Please help me.
What i missed, what is the problem with my query?
How can i replace text in blob?
Depends what you mean by "replace"; using replace to show modified text in select:
select replace(slogen, 'bananas', 'apples') from tcountriesnew where slogen like '%bananas%';
Or update data in a table:
update tcountriesnew set slogen=replace(slogen, 'bananas', 'apples') where slogen like '%bananas%';
BTW. Why are you using blob for text? You should use text type for textual data and blob for binary data.
In some cases it is needed to save texts BLOB. In my case, for whatever reason Drupal 7 developers choose to use a blob for all TEXT columns - this is out of developer's control.
To convert a blob to text, use the MySql convert function. Then you have to save it into the database and convert it again to blob - but this is handled automatically by MySQL. So the following query would do the trick:
UPDATE tcountriesnew
SET slogen = replace(CONVERT(slogen USING utf8), 'bananas', 'apples')
WHERE slogen LIKE '%bananas%';
On MySQL 5.5, This resolved my issue completely.
Also, configure your PhpMyAdmin to show Blob data
Which version are you using ? Maybe it's this bug : http://bugs.mysql.com/bug.php?id=27.
Otherwise try to cast your blob column.
I found a CSV database of Cities/ZIP/GPS, and when I imported it, it added a " infront of the columns.
alt text http://www.grabup.com/uploads/58754a865eebd94c9aafaf7444b52d15.png?direct
I don't want to go in for 33,000 entries and do this manually, is there a query I can run that will remove the quotes?
i'm not a MySql expert but this should work: (based on my similar experience in Sql Server)
UPDATE table_name SET col_name = REPLACE(col_name, '"', '')
For more info on the REPLACE and other string parsing functions, see here:
http://dev.mysql.com/doc/refman/5.0/en/string-functions.html#function_replace
in sql server you coud do:
update mytable set state= substring(state,2,29)
change the "29" to whatever the actual length is.
I am sure mysql must have equivalent syntax.
Repeat for each field, it looks like there is only a handful of them.
As an alternative you could filter the original csv document - isn't that easier?