I imported a MySQL table dump from my server to my local system. I am using phpMyAdmin to view my local MySQL databases. The column that held emails shows as BLOB - instead of actual email.
When I press on "edit" it shows correct emails, but not on listing.
I am totally confused as why this happens. Can anyone suggest solution?
BLOB data type is meant to store arbitrary binary data (Binary Large OBject = BLOB), for example an image or another document. It does not make sense to show the value as-is. It would look the same as viewing an image in a text editor.
You have several options, depending on the version of phpMyAdmin you are using - which I unfortunately do not know.
Make phpMyAdmin show BLOB values by default.
Show BLOB values for a complete result set.
These two possibilities are covered by an already asked question.
But basically, this is just fighting symptoms instead of curing the disease. The question is: Why did you chose the email field to be a BLOB? Basically, a VARCHAR is enough. I do not know the MySQL version you are running, but since MySQL 5.0.3, a VARCHAR can be as large as 65k byte.
ALTER TABLE
`table`
CHANGE
`email` `email` VARCHAR(255) CHARACTER SET utf8 COLLATE utf8_general_ci NOT NULL;
The query above changes the field email to a VARCHAR(255). Pay attention to the length you like to use, and to the character set and collation. But UTF-8 should be very fine in this case.
It shows the text of "Blob" as a placeholder as the blob datatype (which is the datatype that your emails are stored in) is vastly variable in size. For large Blob sizes, if the data were displayed, it could take up 100s + screens to display.
Related
[Edit: my mistake, read the wrong column for the type, the one I'm inserting into is LONGTEXT, not varchar(190)]
I am working with an application that stores the majority of its information in a MySQL database (MySQL server 5.7). One particular value I'm looking at has a 255 character limit enforced by the GUI but, when I looked at that column in the table where it's stored, it's set to varchar(190). I confirmed that I can enter 255-character values in the GUI and that they are not truncated, as I expected.
How can a varchar(190) column store >190 characters? Are there any consequences to doing it this way?
I read 11.4.1 The CHAR and VARCHAR Types and it states that anything over the limit should be truncated.
The answer is that I can't. Misread the column types because that column is varchar(255) when the application builds the schema in PostgreSQL. It's longtext* in MySQL, which explains why I was able to get past the 190 characters. I tried inserting my 230 char test string into the varchar(190) column and it throws an error, as expected.
Need more coffee.
*not sure why longtext, when the application GUI limits input to 255 characters, but I'll need to ask the people who built it.
I came up with the weird situation with my database. My site is already in production and there are lots of data. I need to export CSV file for the client and give them CSV file. I am using my database export button to export my data in CSV file.
My Problem
While exporting CSV file the polish characters are the unreadable character like ą,ć,ó,ś.Means these characters are changed to an unreadable character like "?". Are lots of data so is impossible to change in CSV file manually.
Mistake
I mistakenly set my character set "latin1" and collation "latin1_swedish_ci".
Question
Is it possible to get back this unreadable character into the readable format without losing these data?
I try to googled and try something like https://nicj.net/mysql-converting-an-incorrect-latin1-column-to-utf8/ but no luck.
Any solution?
Thanks in advance.
Reading that the Queston asker wants to know how to update columns to the correct character set in PHPMyAdmin:
1) In PhpmyAdmin Select the table you want to update.
2) Select "Operations" from the top menu
3) Under "Table Options" you want to select the Collation you want to use,
4) And also tick the box stating
Change all column collations
5) Click Go.
All your table data will now be reformed into full UTF-8.
NOTE:
This means that each "char" in a "varchar" is now equal to 4 chars so it can cause some issues with using indexed column keys prior to MySQL 5.7 as indexes were limited to 767 bytes and UTF-8 characters are 4 bytes each -- this an indexed VARCHAR column of 255 characters would be 255 * 4 = 1020 and so exceed the index limit.
I tried to google this, but any results I found were related to importing data from a txt file to populate the database as opposed to storing data.
To me, it seems strange that the contents of a file should be stored in a database. We're working on building an eCommerce site, and each item has a description. I assumed the standard would be to store the description in a txt file and the URL in the database, and not to store the huge contents in the database to keep the file size low and speeds high. When you need to store images in a database, you reference it using a URL instead of storing all the pixel data - why would text be any different?
That's what I thought, but everyone seems to be arguing about VARCHAR vs TEXT, so what really is the best way to store text data up to 1000 characters or so?
Thanks!
Whether you store long text data or image data in a database or in external files has no right or wrong answer. There are pros and cons on both sides—despite many developers making unequivocal claims that you should store images outside the database.
Consider you might want the text data to:
Allow changes to be rolled back.
Support transaction isolation.
Enforce SQL access privileges to the data.
Be searchable in SQL when you create a fulltext index.
Support the NOT NULL constraint, so your text is required to contain something.
Automatically be included when you create a database backup (and the version of the text is the correct version, assuring consistency with other data).
Automatically transfer the text to replica database instances.
For all of the above, you would need the text to be stored in the database. If the text is outside the database, those features won't work.
With respect to the VARCHAR vs. TEXT, I can answer for MySQL (though you mentioned VARCHAR(MAX) so you might be referring to Microsoft SQL Server).
In MySQL, both VARCHAR and TEXT max out at 64KB in bytes. If you use
a multibyte character set, the max number of characters is lower.
Both VARCHAR and TEXT have a character set and collation.
VARCHAR allows a DEFAULT, but TEXT does not.
Internally the InnoDB storage engine, VARCHAR and TEXT are stored identically (as well as VARBINARY and BLOB and all their cousins). See https://www.percona.com/blog/2010/02/09/blob-storage-in-innodb/
Let's say that I have a html form (actually I have an editor - TinyMCE) which through PHP inserts a bunch of text into Mysql table.
I want to know the following:
If I have TINYTEXT data type in Mysql column - what happens if the user tries to put more text than 255 bytes into Mysql table??
Does the application save first 255 bytes and "cuts off" the rest? Or does nothing get saved into Mysql table and mysql issues a warning?? Or none of the above?
Actually, what I want and intend to do is the following:
Limit the size of user form input by setting the column data type in Mysql to TEXT data type, which can hold maximum of 64 KB of text. I want to limit the amount of text that gets passed from user to database, so that user can't put too much data to the server at once.
So, basically, I want to know what happens, if the user puts more text through TinyMCE editor than 65535 bytes, assuming TEXT data type in mysql table.
MySQL, by default, truncates the data if it's too long, and sends a warning.
SHOW WARNINGS;
Data truncated for foo ..
Just to be clear: the data will be saved, but you will be missing the part that was too large.
Default mysql configuration truncate the data if the value is greater than the maximum table field definition size, this will produce a non blocking warning.
If you want a blocking error you have to set the sql_mode to STRICT_ALL_TABLES
dev.mysql.com/doc/refman/5.0/en/server-sql-mode.html#sqlmode_strict_all_tables
IMHO the best way is to manage this error via applicatin software.
Hope this helps
If you enter too much data to a TEXT field in MySQL it will insert the row anyway but with that field truncated to the maximum length, and issue a warning.
Even if MySQL did prevent the row from being added it would not be a good way of limiting the length of data that a user can enter. You should check the length of the POSTed string in PHP, and not run the query at all if it is too long - and perhaps tell the user why their data wasn't entered.
As well as this you can prevent the user from entering too many characters at the client side (although you should always do the check server side as well because someone could bypass the client side limit). It appears that there is no built-in way of doing this in TinyMCE, but it is possible by writing a callback: Limit the number of character in tinyMCE
I am running a production application with MySQL database server. I forget to set column's collation from latin to utf8_unicode, which results in strange data when saving to the column with multi-language data.
My question is, what will happen with my existing data if I change my collation to utf8_unicode now? Will it destroy or corrupt the existing data or will the data remain, but the new data will be saved as utf8 as it should?
I will change with phpMyAdmin web client.
The article http://mysqldump.azundris.com/archives/60-Handling-character-sets.html discusses this at length and also shows what will happen.
Please note that you are mixing up a CHARACTER SET (actually an encoding) with a COLLATION.
A character set defines the physical representation of a string in bytes on disk. You can make this visible, using the HEX() function, for example SELECT HEX(str) FROM t WHERE id = 1 to see how MySQL stores the bytes of your string. What MySQL delivers to you may be different, depending on the character set of your connection, defined with SET NAMES .....
A collation is a sort order. It is dependent on the character set. For example, your data may be in the latin1 character set, but it may be ordered according to either of the two german sort orders latin1_german1_ci or latin1_german2_ci. Depending on your choice, Umlauts such as ö will either sort as oe or as o.
When you are changing a character set, the data in your table needs to be rewritten. MySQL will read all data and all indexes in the table, make a hidden copy of the table which temporarily takes up disk space, then moves the old table into a hidden location, moves the hidden table into place and then drops the old data, freeing up disk space. For some time inbetween, you will need two times the storage for that.
When you are changing a collation, the sort order of the data changes but not the data itself. If the column you are changing is not part of an index, nothing needs to be done besides rewriting the frm file, and sufficiently recent versions of MySQL should not do more.
When you are changing a collation of a column that is part of an index, the index needs to be rewritten, as an index is a sorted excerpt of a table. This will again trigger the ALTER TABLE table copy logic outlined above.
MySQL tries to preserve data doing this: As long as the data you have can be represented in the target character set, the conversion will not be lossy. Warnings will be printed if there is data truncation going on, and data which cannot be represented in the target character set will be replaced by ?
Running a quick test in MySQL 5.1 with a VARCHAR column set to latin1_bin I inserted some non-latin chars
INSERT INTO Test VALUES ('英國華僑');
I select them and get rubbish (as expected).
SELECT text from Test;
gives
text
????
I then changed the collation of the column to utf8_unicode and re-ran the SELECT and it shows the same result
text
????
This is what I would expect - It will keep the data and the data will remain rubbish, because when the data was inserted the column lost the extra character information and just inserted a ? for each non-latin character and there is no way for the ???? to again become 英國華僑.
Your data will stay in place but it won't be fixed.
Valid data will be properly converted:
When you change a data type using
CHANGE or MODIFY, MySQL tries to
convert existing column values to the
new type as well as possible. Warning:
This conversion may result in
alteration of data.
http://dev.mysql.com/doc/refman/5.5/en/alter-table.html
... and more specifically:
To convert a binary or nonbinary
string column to use a particular
character set, use ALTER TABLE. For
successful conversion to occur, one of
the following conditions must
apply:[...] If the column has a
nonbinary data type (CHAR, VARCHAR,
TEXT), its contents should be encoded
in the column character set, not some
other character set. If the contents
are encoded in a different character
set, you can convert the column to use
a binary data type first, and then to
a nonbinary column with the desired
character set.
http://dev.mysql.com/doc/refman/5.1/en/charset-conversion.html
So your problem is invalid data, e.g., data encoded in a different character set. I've tried the tip suggested by the documentation and it basically ruined my data, but the reason is that my data was already lost: running SELECT column, HEX(column) FROM table showed that multibyte chars had been inserted as 0x3F (i.e., the ? symbol in Latin1). My MySQL stack had been smart enough to detect that input data was not Latin1 and convert it into something "compatible". And once data is gone, you can't get it back.
To sum up:
Use HEX() to find out if you still have your data.
Make your tests in a copy of your table.
My question is, what will happen with my existing data if I change my
collation to utf8_unicode now?
Answer: If you change to utf8_unicode_ci, nonthing will happen to your existing data (which is already corrupt and remain corrupt till you modify it).
Will it destroy or corrupt the existing data or will the data remain,
but the new data will be saved as utf8 as it should?
Answer: After you change to utf8_unicode_ci, existing data will not be destroyed. It will remain the same like before (something like ????). However, if you insert new data containing Unicode characters, it will be stored correctly.
I will change with phpMyAdmin web client.
Answer: Sure, you can change collation with phpMyAdmin by going to Operations > Table options
CAUTION! Some problems are solved via
ALTER TABLE ... CONVERT TO ...
Some are solved via a 2-step process
ALTER TABLE ... MODIFY ... VARBINARY...
ALTER TABLE ... MODIFY ... VARCHAR...
If you do the wrong one, you will have a worse mess!
Do SELECT HEX(col), col ... to see what you really have.
Study this to see what case you have: Trouble with utf8 characters; what I see is not what I stored
Perform the correct fix, based on these cases: http://mysql.rjweb.org/doc.php/charcoll#fixes_for_various_cases