I have searched online for a solution and none of the solutions that I found online seems to work here.
My problem is that I have a MySQL dump (SQL file) containing hebrew text values that is saved in UTF8 encoding, and when I import it using "source [file]" it saves the hebrew characters as question marks (???).
Now, when I look at the SQL file (cat [file]) I can see the hebrew characters properly.
Even when I try to copy & paste the SQL commands from the output that "cat" gave directly into the MySQL command line, it works as well.
It only fails when I use "SOURCE [file]" (which I need, because it is a HUGE file).
I have also tried the following:
mysql -uroot -p[pass] --default-character-set=utf8 , and then "SET NAMES utf8" and then "SOURCE [file]" - Gives question marks.
Login to mysql client, then do "SET NAMES utf8", "SET COLLATE utf8_bin" (this is the settings for all the tables in the DB) - Gives question marks.
CREATE DATABASE [db_name] DEFAULT CHARACTER SET UTF8 with the previous setting (section 2 above) - Gives question marks.
mysql -uroot -p[pass] --default-character-set=utf8 [db_name] < [file.sql] - Gives question marks.
set character_set_filesystem utf8 and then running source [file] - Gives question marks.
None of these works properly, the ONLY thing that works is if I do copy+paste directly from cat's output to mysql command line, which is not an option because of the length of the file (several hundreds of MB).
Please help, thanks!
Will be problem reading the file as < file.sql.
As documented about using MySQL in batch mode
If you are running mysql under Windows and have some special characters in the file < file.sql might cause problems, use this instead:
mysql -e "source file.sql" dbname ... --default-character-set=UTF8
Ok, I managed to solve this problem by creating a PHP "import" script.
First, I took the entire SQL file and split it into two files: structure (commands that create the tables and structure) and data.
Then, I just ran the entire structure.sql file using mysql_query, and took the data file, explode it by "\n" to get the seperate lines of all the INSERTs, and then ran them using a loop and mysql_query.
I didnt even need to include "SET NAMES utf8", figures out once I removed this line everything worked perfectly.
I know this isn't an ideal solution, but it is one that worked for me.
Related
I'm having a serious problem with restoring data from a mysqlpump export (note: not mysqldump).
Situation:
MySQL 8.0.19 running on FreeBSD 12.1 running on ESXi.
I have made a clone of that machine.
So they are fully identical in OS and MySQL version and settings (except IP of course).
Let's call the first machine "source" and the second "target".
I run a full DB backup like this:
mysqlpump --set-gtid-purged=ON -u root -p DBName --result-file=DBName.sql
I copy the resulting DBName.sql file to the target host, and import it into mysql by using the command line client and source command in it.
Some of the tables have now corrupted gibberish data in it (collation of fields:utf8mb4_bin).
The structure of the table is fully correct (including the collation of the fields).
But if I export only the specific table with a command like this:
mysqlpump --set-gtid-purged=ON -u root -p DBName TABLEName --result-file=TABLEName.sql
I copy this to the target and import it exactly the same way, everything is correct.
I already spent one full day debugging this, as the datasets involved are massive it's not an easy task.
Anybody has a hint for me what could be the cause of this, how to resolve it or any approach to efficiently debug this?
Thanks!
You can use the parameter --default-character-set-set to specify the exported character set, and then try again.
I want to convert my database to utf-8. What I have done up to now is set the server to read utf-8 and the database is converted by using this query:
ALTER DATABASE database_name CHARACTER SET utf8 COLLATE utf8_unicode_ci;
Now all new information is seen and the things that were broken are now fine. The problem is that the old data is seen as �. This, in my opinion, is due to the fact that the old data is written in windows-1251 (I think at least and I am not 100% sure).
I found out that I need to dump the data:
mysqldump -uroot -p database -r utf8.dump
and then import it:
mysql -uroot -p --default-character-set=utf8 database
mysql> SET names 'utf8'
mysql> SOURCE utf8.dump
This is what I saw from here: https://makandracards.com/makandra/595-dumping-and-importing-from-to-mysql-in-an-utf-8-safe-way
The problem is that I have absolutely no idea where and how to do this.
All I have access is to the web hosting control panel and I have not set up anything on my computer. Therefore, I have no idea how to connect the database to the command shell and so on. What next steps should I do to convert the data to utf-8? Please, any detailed explanation would be great due to the fact that this is the first time for me doing something like this.
// I have a Mac and a Windows machine, but not a Linux at the moment.
Thank you!
The charset and collation of the database are the default for any subsequently created tables. The table setting are the defaults for columns.
For each table, so this:
ALTER TABLE table_name CONVERT TO utf8mb4;
Well I'm not much of a good developer or a database expert. But I have a little understanding of these things. I'm trying to dump a database on a VPS using "mysqldump" command which works perfectly. But when I tried to restore locally after downloading the dump, gives me a time out error.
Can anyone advise me how to dump a database by splitting it into tables separately. The database I'm referring to is pretty large (6 - 7 GB). I actually tried searching and it confuses me.. even this link here confuses me as where to start.
Any help is highly appreciated.
Are you restoring with phpmyadmin? If you try to upload the import it is probably too large.
You can set a directory where the backup files are stored, then you can select the file in phpmyadmin without uploading it.
For the timeout with importing you can increase the timeout settings, or use something like "BigDump"
If you're using mysqldump I'll assume you're familiar with the command line.
On your local machine, use
mysql -u [root] -p [database_name] < [database_dump.sql] -v
enter password: ********
The empty database needs to be created your local machine first before you can import the structure and data to it (as simple as doing CREATE DATABASE [database_name];)
The -v flag will do it in 'verbose' mode so you can see the queries as they run. Omitting '-v' will stop it filling your window with the queries but will also give you that 'is it working or not?' nervous feeling after a few minutes.
This will work on Windows as well as it works on Linux / Mac / anything else
Replace my [placeholders] with your own values.
Thank you so much for all your answers! Well, what I was looking for is a dumping method or a similar script to dump the database table by table. Finally I tried the dumping the output file with a .txt extension which returned me with success.
Below is the command I used (I know its pretty long proceess, but I finally got all tables dumped);
mysqldump -u users -p database_name table_name > table_name.txt
I used the current directory to output the file assuming I'm already in the directory where I need to dump. If you need to dump the output file to a specific dir, then use /path/to/the/dump/table_name.txt instead of just mentioning the table name. Ans make sure you don't enter password after -p. I don't know why, but I left it blank and it prompts for the password. Then when I type password it dumps to a text file.
I hope this helps.!
Once again thank you so much for the users who came in the first place to help me. :)
I want to import data from multiple text files into MySQL. I was thinking of using mysqlimport. However, my lines begin with some prefix, and it turns out mysqlimport does not support the LINES STARTING BY clause of LOAD DATA; apparently this is a WONTFIX bug.
So, what do I do? Can I do better than pipe everything through cut to kill the prefix?
I'm working several years now with MySQL and I must admit, I didn't know of mysqlimport :)
Since you seem to insist on using mysqlimport, you want to stay on command line / console, I guess?
Then you can also just tell MySQL to execute one command without entering interactive mode.
mysql -uYourUserName -pYourPassword -hYourHost DB_Name -e "LOAD DATA INFILE..."
I've been pulling my hear out over this problem for a few hours yesterday:
I've a database on MySQL 4.1.22 server with encoding set to "UTF-8 Unicode (utf8)" (as reported by phpMyAdmin). Tables in this database have default charset set to latin2. But, the web application (CMS Made Simple written in PHP) using it displays pages in utf8...
However screwed up this may be, it actually works. The web app displays characters correctly (mostly Czech and Polish are used).
I run: "mysqldump -u xxx -p -h yyy dbname > dump.sql". This gives me an SQL script which:
looks perfect in any editor (like Notepad+) when displaying in UTF-8 - all characters display properly
all tables in the script have default charset set to latin2
it has "/*!40101 SET NAMES latin2 */;" line at the beginning (among other settings)
Now, I want to export this database to another server running on MySQL 5.0.67, also with server encoding set to "UTF-8 Unicode (utf8)". I copied the whole CMS Made Simple installation over, copied the dump.sql script and ran "mysql -h ddd -u zzz -p dbname < dump.sql". After that, all the characters are scrambled when displaying CMSMS web pages.
I tried setting:
SET character_set_client = utf8;
SET character_set_connection = latin2;
And all combinations (just to be safe, even if it doesn't make any sense to me): latin2/utf8, latin2/latin2, utf8/utf8, etc. - doesn't help. All characters still scrambled, however sometimes in a different way :).
I also tried replacing all latin2 settings with utf8 in the script (set names and default charsets for tables). Nothing.
Are there any MySQL experts here who could explain in just a few words (I'm sure it's simple after all) how this whole encoding stuff really works? I read 9.1.4. Connection Character Sets and Collations but found nothing helpful there.
Thanks,
Matt
Did you try adding the --default-character-set=name option, like this:
mysql --default-character-set=utf8 -h ddd -u zzz -p dbname < dump.sql
I had that problem before and it worked after using that option.
Hope it helps!
Ugh... ok, seems I found a solution.
MySQL isn't the culprit here. I did a simple dump and load now, with no changes to the dump.sql script - meaning I left "set names latin2" and tables charsets as they were. Then I switched my original CMSMS installation over to the new database and... it worked correctly. So actually encoding in the database is ok, or at least it works fine with CMSMS installation I had at my old hosting provider (CMSMS apparently does funny things with characters encoding).
To make it work on my new hosting provider, I actually had to add this line to lib/adodb/drivers/adodb-mysql.inc.php in CMSMS installation:
mysql_query('set names latin2',$this->_connectionID);
This is a slightly modified solution from this post. You can find the exact line there as well. So it looks like mysql client configuration issue.
SOLUTION for me:
set this option in your php file, after mysql_connect (or after mysql_select_db)..
mysql_query("SET NAMES 'utf8'");